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EXECUTIVE  SUMMARY 


The  body  of  this  document  is  a  concise  description  of  suggested  procedures  for 
meeting  the  Technology  Readiness  Assessment  (TRA)  requirements  of  the  Defense  Acqui¬ 
sition  System  (DAS).  The  intent  is  to  provide  the  staff  of  the  Deputy  Under  Secretary  of 
Defense  for  Science  and  Technology  (DUSD(S&T))  a  working  appreciation  of  the  overall 
TRA  process,  with  enough  detail  to  allow  them  to  meet  their  staff  responsibilities.  The 
potential  benefit  to  other  Office  of  the  Secretary  of  Defense  (OSD)  and  Service  Component 
participants  is  recognized.  This  deskbook  should  give  those  involved  with  TRAs  a  greater 
understanding  of  how  TRAs  fit  into  defense  acquisition  and  what  is  expected  by  the 
DUSD(S&T).  The  DUSD(S&T)  serves  as  the  staff  proponent  for  TRAs  for  the  Director 
of  Defense  Research  and  Engineering  (DDR&E). 

The  recently  revised  Department  of  Defense  (DoD)  acquisition  system  is  docu¬ 
mented  in  the  following  documents,  each  of  which  is  available  at  http  ://dod5000 .dau.mil/: 

•  Department  of  Defense  Directive  5000.1,  The  Defense  Acquisition  System, 
dated  May  12,  2003.  This  document  is  referred  to  as  DoDD  5000.1.  It  states 
basic  policy  for  defense  acquisition. 

•  Department  of  Defense  Instruction  5000.2,  Operation  of  the  Defense  Acquisi¬ 
tion  System,  dated  May  12,.  2003.  This  document  is  referred  to  as  DoDI 
5000.2.  It  establishes  a  flexible  management  framework  for  acquisition  pro¬ 
grams  and,  among  other  things,  a  requirement  for  TRAs. 

•  Interim  Defense  Acquisition  Guidebook ,  dated  October  30,  2002.  This  docu¬ 
ment  is  herein  referred  to  as  Interim  Guidebook.  It  provides  nonmandatory 
guidance  drawn  from  the  earlier  DoD  Regulation  5000.2-R. 

A  central  theme  of  the  acquisition  process  is  that  the  technology  employed  in  system 
development  should  be  “mature”  before  system  development  begins.  Normally,  for 
technology  to  be  considered  mature,  it  must  have  been  applied  in  a  prototype  article  (a  sys¬ 
tem,  subsystem,  or  component),  tested  in  a  relevant  or  operational  environment,  and  found  to 
have  performed  adequately  for  the  intended  application.  This  implies  a  need  for  a  way  to 
measure  maturity  and  for  a  process  to  ensure  that  only  sufficiently  mature  technology  is 
employed.  The  Interim  Guidebook  provides  an  outline  of  a  process  and  suggests  activities 
for  performing  TRAs;  however,  this  guidance  is  not  mandatory.  The  document  introduces 
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Technology  Readiness  Levels  (TRLs)  as  an  accepted  way  to  describe  technology  maturity 
and  suggests  activities  that  could  be  carried  out  by  Program  Managers  (PMs),  Component 
Science  and  Technology  (S&T)  Executives,  Component  Acquisition  Executives  (CAEs), 
and  the  DUSD(S&T). 

The  appendixes  provide  extracts  from  relevant  Government  Accounting  Office 
(GAO)  and  DoD  reports;  policy  statements  relevant  to  the  TRA  process;  examples  of  TRLs 
and  TRAs;  specialized  definitions  and  descriptions  of  TRLs  for  software  and  for  drugs, 
vaccines,  and  medical  devices;  example  procedures;  comments  on  Manufacturing  Readiness 
Levels  (MRLs);  and  the  elements  of  a  Technology  Transition  Agreement.  The  expectation  is 
that  the  basic  architecture  of  the  TRA  process  will  remain  relatively  stable  over  time,  whereas 
the  details  implementing  the  process  will  evolve  and  become  more  or  less  explicit  over  time. 
As  changes  occur,  adapting  the  appendixes  or  adding  new  appendixes  will  provide  an  effec¬ 
tive  way  for  the  deskbook  to  accommodate  these  changes. 
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I.  INTRODUCTION 


1.1  BACKGROUND 

The  recently  revised  Department  of  Defense  (DoD)  acquisition  system  is  docu¬ 
mented1  in: 

•  DoD  Directive  5000.1  (DoDD  5000.1),  The  Defense  Acquisition  System ,  dated 
May  12,  2003 

•  DoD  Instruction  5000.2  (DoDI  5000.2),  Operation  of  the  Defense  Acquisition 
System,  dated  May  12,  2003 

•  Interim  Defense  Acquisition  Guidebook  (Interim  Guidebook ),  dated  October 
30,  2002.  This  Interim  Guidebook  contains  nonmandatory  guidance  on  best 
practices,  lessons  learned,  and  expectations.  It  is  anticipated  that  the  Interim 
Guidebook  will  be  revised  in  the  near  future. 

A  central  theme  of  the  acquisition  process  is  that  the  technology  employed  in  system 
development  should  be  “mature”  before  system  development  begins.2  Normally,  for  tech¬ 
nology  to  be  considered  mature,  it  must  have  been  applied  in  a  prototype  article  (a  system, 
subsystem,  or  component),  tested  in  a  relevant  or  operational  environment,  and  found  to 
have  performed  adequately  for  the  intended  application.  This  implies  a  need  for  a  way  to 
measure  maturity  and  for  a  process  to  ensure  that  only  sufficiently  mature  technology  is 
employed. 

DoDI  5000.2  establishes  a  requirement  for  Technology  Readiness  Assessments 
(TRAs),  and  the  Interim  Guidebook  provides  an  outline  of  a  process  for  performing  TRAs. 
The  Interim  Guidebook  introduces  Technology  Readiness  Levels  (TRLs)  as  an  accepted 
way  to  describe  technology  maturity.  The  National  Aeronautics  and  Space  Administration 
(NASA)  has  defined  TRLs  and  has  used  them  in  its  program  reviews,  and  the  NASA  defi¬ 
nitions  are  the  basis  for  the  DoD  definitions.  A  readiness  level  of  TRL  6  or,  preferably,  TRL 


1  All  three  of  the  documents  listed  are  available  at  http://dod5000.dau.mil/. 

2  This  reflects  a  major  conclusion  of  a  study  performed  by  the  General  Accounting  Office  (GAO).  See 
Appendix  A. 
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7  is  normally  achieved  before  a  technology  is  used  in  system  development.3  Section  III  of 
this  document  addresses  TRLs  in  some  detail. 

To  carry  out  TRAs,  the  guidebook  describes  actions  that  would  normally  be  taken 
by  Program  Managers  (PMs),  Component  Science  and  Technology  (S&T)  Executives, 
Component  Acquisition  Executives  (CAEs),  and  the  Deputy  Under  Secretary  of  Defense 
for  Science  and  Technology  (DUSD(S&T)).  TRAs  must  be  carried  out  before  Milestone  B 
and  Milestone  C  of  acquisition  programs  categorized  as  Acquisition  Category  One 
(ACAT  I):  ACAT  ID4  or  ACAT  IAM.5 

1.2  PURPOSE  OF  THIS  DOCUMENT 

This  document  is  intended  to  provide  DUSD(S&T)  staff  participants  a  working 
appreciation  of  the  overall  TRA  process,  with  enough  detail  to  allow  them  to  meet  their  staff 
responsibilities.  The  potential  benefit  to  other  Office  of  the  Secretary  of  Defense  (OSD) 
and  Service  Component  participants  is  recognized.  This  “deskbook”  should  give  those 
involved  with  TRAs  a  greater  understanding  of  how  TRAs  fit  into  defense  acquisition  and 
what  is  expected  by  the  DUSD(S&T).  The  DUSD(S&T)  serves  as  the  staff  proponent  for 
TRAs  for  the  Director  for  Defense  Research  and  Engineering  (DDR&E). 

1.3  ORGANIZATION  OF  THIS  DOCUMENT 

The  body  of  this  document  is  a  concise  description  of  suggested  “best  practices,” 
responsibilities,  roles,  and  procedures  for  meeting  the  TRA  requirements  of  the  Defense 
Acquisition  System  (DAS). 


3  System  development  normally  begins  with  a  Milestone  B  decision. 

4  ACAT  ID  is  a  subcategory  of  ACAT  I.  ACAT  I  programs  are  Major  Defense  Acquisition  Programs 
(MDAPs)  or  programs  that  the  Milestone  Decision  Authority  (MDA)  designates  ACAT  I.  An  MDAP 
is  an  acquisition  program  that  is  not  a  highly  sensitive  classified  program  (as  determined  by  the 
Secretary  of  Defense)  and  is  designated  by  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics  USD(AT&L)  as  an  MDAP  based  on  several  factors  including  research,  development,  test, 
and  evaluation  (RDT&E)  expenditures  and  procurement  expenditures.  The  MDA  for  ACAT  ID 
programs  is  the  USD(AT&L). 

3  ACAT  IAM  is  a  subcategory  of  ACAT  IA.  ACAT  IA  programs  are  Major  Automated  Information 
Systems  (MAISs)  or  programs  designated  by  the  Assistant  Secretary  of  Defense  for  Networks  and 
Information  Integration  (ASD(NII))  (formerly  the  Assistant  Secretary  of  Defense  for  Command, 
Control,  Communications,  and  Intelligence  (ASD(C3I))  to  be  ACAT  IA.  The  MDA  for  ACAT  IAM 
programs  is  the  ASD(NII),  who  is  also  the  DoD  Chief  Information  Officer  (DoD  CIO). 
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The  appendixes  provide  extracts  from  relevant  Government  Accounting  Office 
(GAO)  and  DoD  reports  (see  Appendix  A  and  Appendix  B);  a  policy  statement  relevant  to 
the  TRA  process  (see  Appendix  C);  examples  of  TRLs  (see  Appendix  D);  specialized 
definitions  and  descriptions  of  TRLs  for  software  (see  Appendix  E)  and  for  biomedical 
technology  (see  Appendix  F);  comments  on  Manufacturing  Readiness  Levels  (MRLs)  (see 
Appendix  G);  and  the  elements  of  a  Technology  Transition  Agreement  (see  Appendix  H). 
The  MRLs  are  not  a  part  of  TRAs  and  are  not  yet  implemented,  but  they  are  presented 
because  of  their  similarity  of  purpose  to  TRLs. 


1.4  ACQUISITION  PROCESS  OVERVIEW 

Figure  1-1  shows  the  architecture,  or  framework,  of  the  defense  acquisition  process. 
A  program  to  acquire  a  new  system  or  capability  is  normally  established  in  response  to  a 
recognized  user  need,  but  it  can  also  be  established  to  exploit  a  technological  opportunity 
that  might  result  in  a  new  military  capability,  a  reduced  cost,  or  other  benefit.  Within  this 
framework,  each  program  can  be  structured  to  achieve  the  best  balance  of  cost,  schedule,  and 
performance. 


Process  entry  at  Milestones  A,  B,  or  C 
Entrance  criteria  met  before  entering  phase 

Evolutionary  Acquisition  or  Single  Step  to  Full 
Capability 


A  A  (Program  /\ 

_ /B\lnitiation) _ /c\ 


IOC 


FOC 


Concept 

Technology 

System  Development 

Production  & 

Operations  & 

Refinement 

Development 

&  Demonstration 

Deployment 

Support 

\  Concept 

7  Decision 

s\  Design 
/  \  Readiness 
\ f  Review 

LRIP/IOT&E  A  Decision 

Review 

Pre-Systems  Acquisition 


Systems  Acquisition 


Sustainment 


Figure  1-1.  Defense  Acquisition  Management  Framework  (Source:  DoDI  5000.2) 

The  following  description  of  the  acquisition  system  is  limited  to  the  elements  that 
impact  technology  selection,  development,  and  use  in  defense  system  acquisition. 
DoDI  5000.2  contains  a  far  more  complete  description  of  the  acquisition  system. 
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Consistent  with  a  joint  integrated  architecture,6  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  (USD(AT&L))  leads  “the  development  of  inte¬ 
grated  plans  .  .  ,”7  With  advice  from  the  Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS)  and 
the  Joint  Requirements  Oversight  Council  (JROC),  multiple  DoD  communities  “assist  in 
formulating  broad,  time  phased,  operational  goals,  and  [in]  describing  requisite  capabilities 
in  [an]  Initial  Capabilities  Document  (ICD).”8  After  analysis  of  potential  system  concepts, 
the  ICD  describes  a  selected  concept  based  on  “robust  analyses  that  consider  affordability, 
technology  maturity,  and  responsiveness.”9 

The  ICD  and  a  plan  for  an  Analysis  of  Alternatives  (AoA)  are  presented  to  the  Mile¬ 
stone  Decision  Authority  (MDA)  for  approval.  Approval  initiates  Concept  Refinement. 
During  Concept  Refinement,  the  selected  concept  is  refined,  and  alternative  technologies 
(not  alternative  concepts)  are  analyzed.  This  analysis  includes  consideration  of  the  maturity 
of  the  alternative  technologies.  Whenever  the  system  concept  requires  technologies  that  are 
promising  but  still  unproven,  the  Component  includes  a  project  for  maturing  the  technology 
in  a  Technology  Development  Strategy  (TDS).  Among  other  things,  the  TDS  describes  how 
the  program  will  be  divided  into  technology  spirals  and  development  increments.  The  pro¬ 
gram  enters  Technology  Development  (TD)  at  Milestone  A  when  the  MDA  approves  the 
TDS.  The  program  is  still  not  considered  an  acquisition  program.10 

During  TD,  the  technologies  required  to  design  and  build  the  system  are  pursued  so 
that  they  will  be  sufficiently  mature  by  Milestone  B.  TD  is  a  continuous  technology  discov¬ 
ery  and  development  process  that  reflects  a  close  collaboration  between  the  user  and  the 
system  developer  and  between  the  system  developer  and  the  technology  developers.11  This 
phase  reduces  technology  risk  and  determines  which  technologies  are  mature  and  should  be 
integrated  into  a  system.  For  an  evolutionary  program,  this  selection  of  mature  technologies 


6  The  joint  integrated  architectures  are  developed  collaboratively  by  the  USD(AT&L),  the  ASD(NII) 
(formerly  ASD(C3I),  the  Joint  Staff,  the  Military  Departments,  the  Defense  Agencies,  Combatant 
Commanders,  and  other  appropriate  DoD  Components.  See  DoDI  5000.2,  paragraph  3.2. 1.1. 

7  DoDI  5000.2,  paragraph  3.2.2. 

8  DoDI  5000.2,  paragraph  3.4.1. 

9  DoDI  5000.2,  paragraph  3.4.1. 

10  Shipbuilding  acquisition  programs  can  be  initiated  at  Milestone  A.  See  DoDI  5000.2,  paragraph  3.6.3. 

1 1  The  system  developer  and  the  technology  developers  may  formalize  their  association  with  Technology 
Transition  Agreements.  Appendix  J  contains  an  example  template  for  an  agreement. 
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applies  to  the  next  increment  that  will  have  a  Milestone  B.  TD  continues  for  subsequent 
increments,  each  of  which  has  its  own  Milestone  B. 

A  TRA  must  be  conducted  before  each  Milestone  B  (and  before  each  Milestone  C). 
One  of  the  criteria  for  exiting  TD  is  that  the  technology  has  been  demonstrated  in  a  relevant 
environment.12  TD  demonstrations  are  used  to  substantiate  technology  maturity.  These 
demonstrations  should  use  prototypes  or  engineering  development  models  (EDMs)  at  the 
subsystem  level.  That  is,  these  items,  after  detailed  design,  should  be  suitable  for  integration 
into  the  system. 

During  the  TD  phase,  the  Joint  Staff  produces  a  Capability  Development  Document 
(CDD)  that  builds  on  the  ICD  and  supports  the  initiation  of  an  acquisition  program.  The 
CDD  provides  the  detailed  operational  performance  parameters  necessary  to  design  the 
proposed  system. 

The  technologies  chosen  for  the  system  must  provide  an  affordable  increment  of 
capability.13  This  requires  that  the  chosen  technologies  are  producible  at  an  acceptable  cost 
and  production  rate.  While  not  explicit  in  DoDI  5000.2,  this  implies  that  manufacturability 
and  producibility  have  been  considered  in  the  selection  of  technologies. 

Milestone  B  authorizes  a  program  or  increment  of  a  program  to  enter  System 
Development  and  Demonstration  (SDD).  SDD  consists  of  two  major  efforts  (System  Inte¬ 
gration  and  System  Demonstration)  and  a  mid-phase  Design  Readiness  Review  (DRR). 
System  Integration  is  the  system  design  phase  during  which  the  chosen  technologies  and 
subsystems  are  integrated  into  a  detailed  system  design,  and  the  manufacturing  processes 
are  developed.  This  effort  typically  includes  demonstration  of  prototype  articles  or  EDMs 
that  result  from  integration  of  some  or  all  of  the  subsystems.  The  DRR  marks  the  transition 
to  System  Demonstration.  During  System  Demonstration  prototypes  are  demonstrated  in 
the  intended  environment,  showing  that  the  system  can  meet  approved  requirements.14  This 


12  DoDI  5000.2,  paragraph  3.6.7. 

13  DoDI  5000.2,  paragraph  3.6.7. 

14  After  DRR,  a  Capability  Production  Document  (CPD)  is  finalized  by  the  Joint  Staff,  and  it  is  validated 
and  approved  before  Milestone  C.  Key  Performance  Parameters  (KPPs)  from  the  CPD  are  inserted 
verbatim  into  the  acquisition  strategy  and  the  Acquisition  Program  Baseline  (APB).  See  Chairman  of 
the  Joint  Chiefs  of  Staff  Manual  (CJCSM)  3170.01,  Operation  of  the  Joint  Capabilities  Integration 
and  Development  System,  dated  24  June  2003,  Enclosure  F  paragraphs  1.  and  2 
(http://www.teao.saic.com/jfcom/ier/documents/m317001.pdD. 


1-5 


phase  must  also  establish  that  no  significant  manufacturing  risk  exists  and  that  industrial 
capabilities  are  reasonably  available. 

A  new  or  revised  TRA  is  required  before  Milestone  C.  This  TRA  should  reflect  the 
resolution  of  any  technology  deficiencies  that  arose  during  SDD  and  should  establish  that 
all  critical  manufacturing  technologies  are  mature. 

Milestone  C  follows  SDD  and  authorizes  Low  Rate  Initial  Production  (LRIP). 
LRIP  completes  manufacturing  development  to  ensure  efficient  manufacturing  capability 
and  produces  production-representative  articles  for  Initial  Operational  Test  and  Evaluation 
(IOT&E).15 

Approval  for  Full  Rate  Production  (FRP)  depends  on  demonstrating  that  critical 
manufacturing  processes  are  under  control  and  that  statistical  process  control  data  are  being 
collected. 

The  framework  just  described  can  be  tailored  to  a  specific  acquisition  program 
structure.  For  example,  the  program  does  not  have  to  start  at  Concept  Refinement.  It  can 
start  at  any  point  consistent  with  phase- specific  entrance  criteria  and  statutory  requirements. 
If  it  starts  at  or  beyond  Milestone  B,  an  associated  TRA  is  conducted  to  ensure  that  the 
technology  is  ready  for  the  upcoming  phase  of  acquisition.  Normally,  a  program  is  not  con¬ 
sidered  an  acquisition  program  until  it  has  passed  Milestone  B. 

DoDI  5000.2  establishes  evolutionary  development  as  the  strategy  DoD  prefers: 

3.3.2.  The  approaches  to  achieve  evolutionary  acquisition  require 
collaboration  between  the  user,  tester,  and  developer.  They  include: 

3.3.2. 1.  Spiral  Development.  In  this  process,  a  desired 
capability  is  identified,  but  the  end-state  requirements  are  not  known  at  pro¬ 
gram  initiation.  Those  requirements  are  refined  through  demonstration  and 
risk  management;  there  is  continuous  user  feedback;  and  each  increment 
provides  the  user  the  best  possible  capability.  The  requirements  for  future 
increments  depend  on  feedback  from  users  and  technology  maturation. 

3. 3. 2. 2.  Incremental  Development.  In  this  process,  a  desired 
capability  is  identified,  an  end-state  requirement  is  known,  and  that  require¬ 
ment  is  met  over  time  by  developing  several  increments,  each  dependent  on 
available  mature  technology. 


From  DoDI  5000.2,  3. 8. 3. 4.  “LRIP  is  not  applicable  to  AISs  or  software-intensive  systems  with  no 
developmental  hardware;  however  a  limited  deployment  phase  may  be  applicable.  Software  shall  have 
demonstrated  the  maturity  level  required  in  the  CPD  before  deploying  it  to  the  operational 
environment.”  An  AIS  is  an  automated  information  system. 
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For  hardware  systems,  evolutionary  development  normally  uses  incremental  devel¬ 
opment.  Each  successive  design  unit  is  called  an  increment  (Increment  1,  Increment  2,  and 
so  forth).  To  ensure  that  the  technology  is  mature,  a  TRA  is  required  for  each  increment 
before  the  program  has  a  Milestone  B  or  Milestone  C  review  for  that  increment.16 

Software  is  normally  developed  using  the  spiral  development  process.  This  is  an 
iterative,  cyclical  process  of  build-test-fix-test-deploy.  Each  release  builds  on  the  lessons  of 
the  previous  release.  There  can  be  several  releases  during  the  acquisition  and  deployment  of 
a  system  or  system  increment.  In  the  TRA  process,  software  is  considered  an  integral  part 
of  the  system  or  subsystem  in  which  it  operates.  Therefore,  demonstration  of  a  technology 
at  the  subsystem  or  system  level  must  include  demonstration  of  the  associated  software.  The 
Army,  for  its  use,  has  defined  TRLs  for  software  (see  Appendix  G). 


16  DoDI  5000.2,  paragraph  3.7.2.4  and  Table  E3.T2. 
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II.  KEY  ACTIVITIES  AND  RELATIONSHIPS 


Much  of  the  material  in  the  following  paragraphs  is  based  on  the  Interim  Guide¬ 
book,  however,  the  responsibilities  and  processes  in  the  guidebook  (which  is  based  on  DoD 
5000. 2-R,  now  canceled) 17  are  not  mandatory.  Therefore,  the  following  is  a  “suggestion” 
of  activities  and  relationships  that  can  accomplish  the  required  TRAs. 

Before  an  acquisition  program  can  enter  SDD  (at  Milestone  B)  or  LRIP  (at  Mile¬ 
stone  C),  technology  maturity  must  be  assessed.18  DoDI  5000.2,  paragraph  3.7.2,  estab¬ 
lishes  as  acquisition  policy  that  “. . .  Unless  some  other  factor  is  overriding  in  its  impact,  the 
maturity  of  the  technology  shall  determine  the  path  to  be  followed.”  Paragraph  3. 7. 2. 2 
states  that  “...  If  [the]  technology  is  not  mature,  the  DoD  Component  shall  use  alternative 
technology  that  is  mature  and  that  can  meet  the  user’s  needs.” 

The  PM  is  an  especially  important  figure  in  defense  acquisition.  He/she  is  respon¬ 
sible  for  planning  and  managing  each  program.  The  PM  normally19  reports  to  a  Program 
Executive  Officer  (PEO),  who  oversees  several  PMs.  The  PEO  reports  directly  to  the  CAE, 
who  reports  through  the  Component  Secretary  to  the  USD(AT&L).  Similarly  important  is 
the  Component  S&T  Executive.  He/she  reports  to  the  CAE  and  is  responsible  for  devel¬ 
oping  the  noncommercial  technologies  that  the  Component  will  need  to  meet  future 
operational  requirements.  The  DUSD(S&T)  has  an  oversight  responsibility  for  this  TD 
program  as  part  of  managing  the  overall  S&T  program  within  DoD. 

The  Interim  Guidebook  suggests  that  the  Component  S&T  Executive  should  be 
responsible  for  directing  the  Component  TRAs.  For  ACAT  I  and  ACAT  IA  programs,  these 
TRAs  are  submitted  to  the  CAE  for  approval,  and  an  information  copy  is  sent  to  the 
DUSD(S&T).  Subsequently,  the  CAE  transmits  the  action  copy  of  the  TRA  to  the 
DUSD(S&T),  who  is  responsible  for  evaluating  each  TRA  received  from  a  Component. 


1 7  A  new  document  to  replace  the  Interim  Guidebook  is  being  developed. 

18  This  is  a  regulatory  requirement.  See  DoDI  5000.2,  Table  E3.T2. 

19  For  a  few  special  programs,  the  PM  reports  directly  to  the  CAE. 


The  TRA  process  involves  the  participation  of  the  PM,  the  Component  S&T  Execu¬ 
tive,  and  the  DUSD(S&T).20  Figure  II- 1  is  a  nominal  timeline  for  the  TRA  activities  while 
Figure  II-2  displays  the  principal  activities  of  the  DUSD(S&T)  Action  Officer  (AO). 

The  following  paragraphs  describe  the  key  activities  and  people  involved  in  the  TRA 
process.  Section  IV  of  this  document  explores  the  TRA  process  in  more  detail. 

2.1  PROGRAM  MANAGER  (PM) 

2.1.1  Requesting  Milestone  Review  Meetings 

Most  likely,  a  PM  will  be  designated  during  TD  to  guide  that  development  and  to 
prepare  for  Milestone  B .  The  PM  is  responsible  for  requesting  Milestone  B  and  C  review 
meetings.  For  ACAT  ID  programs,  the  Defense  Acquisition  Board  (DAB)21  conducts  the 
review.  For  ACAT  IAM  programs,  the  Information  Technology  Acquisition  Board 
(ITAB)22  conducts  the  review. 

Concurrently  with  scheduling  a  milestone  review  meeting,  the  PM  establishes  a 
schedule  for  the  submission  of  critical  technologies.  When  establishing  the  schedule  for 
submitting  critical  technologies,  coordination  with  the  Component  S&T  Executive  [and  with 
the  DUSD(S&T)  for  ACAT  ID  and  ACAT  IAM  programs]  is  important  so  that  each  orga¬ 
nization  has  ample  time  to  complete  its  respective  TRA  activities. 

2.1.2  Determining  Critical  Technologies  and  Disseminating  Information 

The  PM  has  a  fundamental  responsibility  to  know  which  technologies  are  critical.  A 
technology  is  “critical”  if  the  system  being  acquired  depends  on  this  technology  to  meet 
capability  thresholds,  with  acceptable  development  cost  and  schedule  and  with  acceptable 
production  and  operation  costs,  and  if  the  technology  or  its  application  is  either  new  or 
novel.  Said  another  way,  a  new  or  novel  technology  is  critical  if  it  is  necessary  to  achieve  the 
successful  development  of  a  system,  its  acquisition,  or  its  operational  utility. 


2(1  Appendix  B  includes  from  DoDD  5000.1,  DoDI  5000.2,  and  the  Interim  Guidebook  extracts  that 
establish  or  suggest  TRA  responsibilities. 

21  The  DAB  is  chaired  by  the  USD(AT&L),  who  is  the  MDA  for  ACAT  ID  programs.  The  Vice 
Chairman  of  the  Joint  Chiefs  of  Staff  (VCJCS)  serves  as  the  vice  chairman. 

22  The  ITAB  is  chaired  by  the  ASD(NII)  (formerly  ASD(C3I),  who  is  the  DoD  CIO  and  MDA  for  ACAT 
IAM  programs. 
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About  16  weeks  before  a  milestone  review  (see  Figure  II- 1),  on  the  schedule  agreed 
to  with  the  DUSD(S&T)  and  the  Component  S&T  Executive,  the  PM  should  identify  the 
critical  technologies  and  compile  the  status,  test  results,  and  other  information  necessary  to 
assess  the  maturity  of  these  technologies.  This  identification  of  critical  technologies  is  an 
important  step  in  the  TRA  process.  For  a  readiness  assessment  to  be  useful,  it  must  include 
all  the  critical  technologies.  Before  identifying  the  critical  technologies,  it  would  be  helpful  if 
the  PM  would  send  the  DUSD(S&T)  and  the  Component  S&T  Executive  a  memorandum 
that  describes  the  identification  process  that  will  be  used. 

After  determining  the  critical  technologies,  the  PM  provides  this  information  to  the 
Component  S&T  Executive  and  sends  an  information  copy  to  the  DUSD(S&T).  Prefer¬ 
ably,  the  identification  of  critical  technologies  will  have  been  vetted  and  agreed  upon 
between  the  PM  and  Component  S&T  Executive.  In  addition  to  the  list  of  critical  technolo¬ 
gies,  the  PM  should  explain  the  function  of  each  technology  in  the  system  and  provide 
information  on  its  status.  This  could  include  records  of  tests  or  applications  of  the  technol¬ 
ogy.  The  PM  should  also  provide  any  additional  information  requested  by  the  Component 
S&T  Executive  or  the  DUSD(S&T). 

If  an  ACAT  ID  or  ACAT  IAM  program  integrates  critical  systems  or  subsystems 
that  are  being  developed  in  other  programs,  the  PM  of  the  higher  order  program,  in  prepara¬ 
tion  for  a  TRA,  should  identify  the  critical  technologies— including  interface  technolo¬ 
gies— used  on  his/her  side  of  the  interfaces.  This  PM  should  request  (through  the 
appropriate  PEO  or  CAE,  as  necessary)  and  obtain  the  identification  of  any  critical  tech¬ 
nologies  in  the  lower  order  programs.  The  critical  technologies  of  both  the  higher  order 
system  and  all  lower  order  systems  or  subsystems  are  included  in  the  list  of  critical  tech¬ 
nologies  the  PM  of  the  higher  order  system  submits  to  his/her  Component  S&T  Executive 
and  the  DUSD(S&T). 

If  a  program  has  competing  designs  at  the  time  of  the  Milestone  B  or  Milestone  C 
review,  the  critical  technologies  of  each  design  should  be  identified  separately. 

2.2  COMPONENT  SCIENCE  AND  TECHNOLOGY  (S&T)  EXECUTIVE 

2.2.1  Providing  the  Required  Technology 

The  Component  S&T  Executive  is  responsible  for  developing  the  noncommercial 
technologies  that  will  be  needed  to  meet  future  operational  requirements.  In  addition  to 
advising  PMs  regarding  the  status  and  applicability  of  technologies,  the  Component  S&T 
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Executive  should  work  with  the  PMs  to  establish  how  technologies  will  be  matured  to  sup¬ 
port  system  development  programs.  During  TD,  before  Milestone  B,  the  Component  S&T 
Executive  and  Component  laboratories  will  likely  be  providing  some  of  the  resources  and 
effort  that  the  PM  has  identified  in  the  TD  strategy. 

2.2.2  Directing  the  TRA 

The  Interim  Guidebook  suggests  that  the  Component  S&T  Executive  should  direct 
the  TRA  and  decide  how  it  will  be  conducted.  The  TRA  must  include  all  critical  technolo¬ 
gies  identified  by  the  PM  and  can  include  additional  technologies  that  the  Component  S&T 
Executive  considers  critical.  Typically,  much  of  the  information  used  in  a  TRA  comes  from 
the  PM;  however,  the  assessment  must  be  independent  of  the  PM. 

The  TRL  definitions  (see  Section  III,  Table  III- 1)  provide  a  convenient  nomenclature 
for  a  technology’s  maturity  status.  The  Component  should  use  TRLs  to  relate  TRA 
findings  unless  alternative  means  have  been  coordinated  beforehand  with  the  DUSD(S&T). 

2.2.3  Processing  the  TRA  Results 

For  ACAT  ID  and  ACAT  IAM  programs,  the  Component  S&T  Executive  signs  the 
TRA  (or  accompanying  memorandum)  and  accepts  responsibility  for  its  accuracy.  He/she 
then  submits  the  TRA  to  the  CAE  and,  at  the  same  time,  sends  an  information  copy  to  the 
DUSD(S&T). 

2.3  COMPONENT  ACQUISITION  EXECUTIVE  (CAE) 

For  ACAT  ID  and  ACAT  IAM  programs,  the  CAE  submits  a  report  to  the 
DUSD(S&T),  with  an  assessed  TRL  (or  some  equivalent  measure)  for  each  critical  tech¬ 
nology.  This  report  can  consist  of  a  cover  letter  or  memorandum  endorsing  the  Component 
TRA  and  officially  transmitting  that  TRA.  This  should  be  accomplished  according  to  the 
agreed-upon  schedule— normally,  at  least  6  weeks  before  a  scheduled  Milestone  B  or  Mile¬ 
stone  C.  See  Figure  II- 1. 
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2.4  DEPUTY  UNDER  SECRETARY  OF  DEFENSE  FOR  SCIENCE  AND 
TECHNOLOGY  (DUSD(S&T)) 

2.4.1  Preparation  and  Oversight 

The  DUSD(S&T)  has  both  oversight  and  evaluation  responsibilities  for  the  TRA. 
An  AO  assists,  as  directed  by  the  DUSD(S&T)  (see  Figure  II-2).  While  the  Component  is 
preparing  the  TRA,  the  AO  reviews  the  critical  technologies  and  the  identification  process, 
negotiates  any  perceived  deficiencies,  and  provides  oversight.  In  addition,  the  AO  partici¬ 
pates  in  the  TRA  to  the  extent  mutually  agreed  upon  with  the  Component  S&T  Executive. 

2.4.2  Evaluating  the  Component  TRA 

The  DUSD(S&T)  evaluates  the  Component  TRA  in  cooperation  with  the  Compo¬ 
nent  S&T  Executive  and  the  PM.  There  is  no  rigid  requirement  that  every  critical  technol¬ 
ogy  be  at  a  pre-specified  TRL  by  Milestone  B  or  Milestone  C.  However,  for  Milestone  B, 
readiness  levels  of  at  least  TRL  6  are  typical  (TRL  7  preferred),  and,  for  Milestone  C,  readi¬ 
ness  levels  of  at  least  TRL  8  are  typical  (TRL  9  preferred).  At  Milestone  B,  the 
DUSD(S&T)  might  conclude  that  a  readiness  level  of  TRL  5  is  adequate  for  a  critical  tech¬ 
nology  if  a  planned  and  funded  program  is  in  place  to  mature  the  technology  quickly  or  if  a 
mature  backup  technology  that  meets  the  program  requirements  and  schedule  exists.  If  the 
Component  expects  such  a  conclusion,  the  supporting  information  should  be  provided 
along  with  the  TRA.  At  Milestone  C,  a  similar  situation  could  arise— most  likely  with 
respect  to  the  manufacturing  process  technology  required  to  achieve  required  production 
rates  or  cost  goals.  Section  III  of  this  document  addresses  TRLs  in  some  detail. 

After  evaluating  the  Component  TRA,  the  DUSD(S&T)  either  concurs  with  the 
findings  or  conducts  an  independent  TRA.  The  DUSD(S&T)  forwards  either  a  concurrence 
with  the  findings  of  the  Component  TRA  or  the  findings  of  the  independent  TRA  to  the 
Overarching  Integrated  Product  Team  (OIPT)  and  the  DAB  or  to  the  IT  OIPT  and  the 
ITAB.  This  takes  place  at  least  15  days  before  a  Milestone  B  or  Milestone  C  review  meeting 
(see  Ligure  II- 1).  If  this  15-day  window  is  not  possible,  the  date  of  the  review  meeting 
should  be  reconsidered  so  the  OIPT  and  DAB  members  or  the  IT  OIPT  and  ITAB  mem¬ 
bers  have  ample  time  to  review  all  the  relevant  information. 
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2.4.3  Preparing  the  National  Defense  Authorization  Act  (NDAA)  Reports  for 
the  Secretary  of  Defense 

Sec.  804  of  the  NDAA  for  Fiscal  Year  2002  Conference  Report  requires  the  Sec¬ 
retary  of  Defense  to  submit  reports  on  the  implementation  of  the  DoD  technology  readiness 
policy.  The  DUSD(S&T)  is  responsible  for  preparing  these  reports.  Paragraph  2.7 
describes  the  responsibilities  and  procedures  in  more  detail. 

2.5  CHAIRMAN,  OVERARCHING  INTEGRATED  PRODUCT  TEAM  (OIPT) 

The  OIPT  [or,  in  the  case  of  an  ACAT  IAM  program,  the  Information  Technology 
Overarching  Integrated  Product  Team  (IT  OIPT)]  is  led  by  the  appropriate  OSD  office.  It  is 
composed  of 

•  The  PM 

•  The  PEO 

•  Representatives  of  the  Component  staff,  the  USD(AT&L)  staff,  the 
ASD(NII)23  staff,  and  the  Joint  Staff 

•  Other  OSD  principals  involved  in  the  oversight  and  review  of  a  particular 
ACAT  ID  or  ACAT  IAM  program. 

The  OIPT  or  IT  OIPT  provides  strategic  guidance  for  the  early  resolution  of  issues  and 
conducts  oversight  and  review  as  a  program  proceeds  through  its  acquisition  life  cycle. 

2.6  MILESTONE  DECISION  AUTHORITY  (MDA) 

The  MDA  is  the  individual  designated  in  accordance  with  criteria  established  by  the 
USD(AT&L)— or  the  ASD(NII)  for  Automated  Information  System  (AIS)  acquisition  pro¬ 
grams— to  approve  the  entry  of  an  acquisition  program  into  the  next  phase.  The  DAB  or 
ITAB  provides  a  recommendation  to  assist  the  MDA  in  the  decision. 

2.7  SECRETARY  OF  DEFENSE 

For  each  of  the  calendar  years  2002  through  2005,  the  Secretary  of  Defense  is 
required  to  report  to  Congress  on  the  implementation  of  DoD  policy  regarding  technology 


23  ASD(NII):  This  position  was  formerly  the  Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications,  and  Intelligence  (ASD(C3I)). 
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maturity  at  the  initiation  of  MDAPs.24  According  to  Sec.  804  of  the  NDAA  for  Fiscal  Year 
2002  Conference  Report,  the  reports  must 


identify  each  case  in  which  a  major  defense  acquisition  program  entered 
system  development  and  demonstration  [i.e.,  passed  Milestone  B]  during  the 
preceding  calendar  year  and  into  which  key  technology  has  been  incorpo¬ 
rated  that  does  not  meet  the  technological  maturity  requirement  [i.e.,  that 
technology  must  have  been  demonstrated  in  a  relevant  environment  or,  pref¬ 
erably,  in  an  operational  environment,  to  be  considered  mature  enough  to  use 
for  product  development  in  systems  integration]  . . .  and  provide  a  justifica¬ 
tion  for  why  such  technology  was  incorporated;  and 

identify  any  determination  of  technological  maturity  with  which  the  Deputy 
Under  Secretary  of  Defense  for  Science  and  Technology  did  not  concur  and 
explain  how  the  issue  has  been  or  will  be  resolved. 

The  report  for  each  calendar  year  must  be  submitted  to  the  Committees  on  Aimed 
Services  of  the  Senate  and  the  House  of  Representatives  by  March  1  of  the  following  year 
(i.e.,  March  1  of  years  2003  through  2006). 

At  the  conclusion  of  each  MDAP  milestone  review,  an  office  designated  by  the 
DUSD(S&T)  will  compile  the  necessary  information  for  these  reports.  At  the  beginning  of 
each  calendar  year  (2003  through  2006),  the  designated  office  will  prepare  the  report  for  the 
Congressional  committees.  The  DUSD(S&T)  will  submit  the  report  through  the  DDR&E 
to  the  USD(AT&L)  for  concurrence  and  forwarding  to  the  immediate  office  of  the  Secre¬ 
tary  of  Defense.  The  Secretary  of  Defense  will  sign  the  report  or  cover  letter  and  submit  it 
to  the  Congressional  committees  as  required. 


24  This  requirement  is  contained  in  Sec.  804  of  the  NDAA  for  Fiscal  Year  2002  Conference  Report. 
Appendix  C  of  this  deskbook  contains  the  complete  text.  The  policy  to  which  the  Conference  Report 
refers  is  in  the  then  current  DoDl  5000.2,  paragraph  4. 7. 3. 2. 2. 2.  In  the  current  version  of  DoDI  5000.2 
(dated  May  12,  2003),  the  corresponding  policy  statement  is  in  paragraph  3. 7. 2. 2.  This  latter  paragraph 
states  “Technology  developed  in  S&T  or  procured  from  industry  or  other  sources  shall  have  been 
demonstrated  in  a  relevant  environment  or,  preferably,  in  an  operational  environment  to  be  considered 
mature  enough  to  use  for  product  development  in  systems  integration.  Technology  readiness 
assessments,  and  where  necessary,  independent  assessments,  shall  be  conducted.  If  [the]  technology  is 
not  mature,  the  DoD  Component  shall  use  alternative  technology  that  is  mature  and  that  can  meet  the 
user’s  needs.” 
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III.  TRL  DEFINITIONS 


The  Interim  Guidebook  establishes  technology  maturity  levels  (i.e.,  TRLs)  as  the 
preferred  descriptor  of  technology  maturity  for  the  TRAs  required  for  ACAT  ID  and 
ACAT  IAM  programs.  Other  means  to  accomplish  a  TRA  are  allowed  but  should  be  coor¬ 
dinated  in  advance  by  the  DUSD(S&T). 

Using  TRLs  to  describe  the  maturity  of  technologies  considered  for  a  new  system 
originated  with  NASA  in  the  early  1980s.  The  levels  ran  from  the  earliest  stages  of  scientific 
investigation  (Level  1)  to  successful  use  in  a  system  (Level  9),  which  equates  to  space  flight 
for  NASA.  DoD  has  adopted  the  NASA  definitions— with  only  minor  modifications— for 
the  nine  TRLs. 

Having  a  strong  grasp  of  the  TRL  concept  is  important.  The  tables  in  this  section 
give  the  TRL  fundamentals.  Table  III- 1  defines  and  describes  the  DoD  TRL  levels.  It  also 
lists  typical  documentation  that  should  be  extracted  or  referenced  to  support  a  TRL  assign¬ 
ment.  Table  III-2  includes  a  set  of  additional  definitions  that  help  provide  a  uniform  inter¬ 
pretation  of  the  levels. 

Software  is  likely  to  be  an  important  element  in  many  TRAs.  Since  the  TRL  defini¬ 
tions  in  Table  III- 1  reflect  a  systems  approach  in  which  software  is  treated  as  a  part  of  a 
component  or  system,  software  TRLs  are  not  spelled  out  specifically  in  these  definitions. 
However,  because  some  guidelines  would  be  useful  in  determining  the  TRLs  of  the  software 
parts  of  components  and  systems,  Appendix  G  provides  a  set  of  software  TRL  definitions 
developed  by  the  Army.25 

The  TRL  definitions  in  Table  III-l  are  not  readily  applied  to  medical-related  items, 
specifically  drugs,  vaccines,  and  medical  devices.  Their  development  and  use  must  adhere  to 
Food  and  Drug  Administration  (FDA)  statutes  and  policy  and  to  DoD  statutes  and  policy. 
The  Army,  in  recognition  of  this  situation,  took  the  initiative  to  establish  biomedical  TRLs 
Appendix  H  provides  the  excellent  result  of  their  efforts. 


25  According  to  the  Interim  Guidebook,  Appendix  6  of  that  guidebook  “lists  the  various  technology 
readiness  levels  and  descriptions  from  a  systems  approach  for  both  HARDWARE  and  SOFTWARE. 
DoD  Components  may  provide  additional  clarifications  for  Software.” 
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Table  111-1.  TRL  Definitions,  Descriptions,  and  Supporting  Information 
(Source:  Interim  Guidebook ,  dated  October  30,  2002) 


TRL 

Definition 

Description 

Supporting  Information 

1 

Basic  principles  observed 
and  reported 

Lowest  level  of  technology 
readiness.  Scientific  research 
begins  to  be  translated  into 
applied  research  and 
development.  Examples  might 
include  paper  studies  of  a 
technology’s  basic  properties. 

Published  research  that 
identifies  the  principles  that 
underlie  this  technology. 
References  to  who,  where, 
when. 

2 

Technology  concept 
and/or  application 
formulated 

Invention  begins.  Once  basic 
principles  are  observed,  practical 
applications  can  be  invented. 
Applications  are  speculative,  and 
there  may  be  no  proof  or  detailed 
analysis  to  support  the 
assumptions.  Examples  are 
limited  to  analytic  studies. 

Publications  or  other  references 
that  outline  the  application  being 
considered  and  that  provide 
analysis  to  support  the  concept. 

3 

Analytical  and 
experimental  critical 
function  and/or 
characteristic  proof  of 
concept 

Active  research  and 
development  is  initiated.  This 
includes  analytical  studies  and 
laboratory  studies  to  physically 
validate  analytical  predictions  of 
separate  elements  of  the 
technology.  Examples  include 
components  that  are  not  yet 
integrated  or  representative. 

Results  of  laboratory  tests 
performed  to  measure 
parameters  of  interest  and 
comparison  to  analytical 
predictions  for  critical 
subsystems.  References  to 
who,  where,  and  when  these 
tests  and  comparisons  were 
performed. 

4 

Component  and/or 
breadboard  validation  in 
[a]  laboratory  environment 

Basic  technological  components 
are  integrated  to  establish  that 
they  will  work  together.  This  is 
relatively  “low  fidelity”  compared 
to  the  eventual  system. 

Examples  include  integration  of 
“ad  hoc”  hardware  in  the 
laboratory. 

System  concepts  that  have 
been  considered  and  results 
from  testing  laboratory-scale 
breadboard(s).  References  to 
who  did  this  work  and  when. 
Provide  an  estimate  of  how 
breadboard  hardware  and  test 
results  differ  from  the  expected 
system  goals. 

5 

Component  and/or 
breadboard  validation  in 
[a]  relevant  environment 

Fidelity  of  breadboard  technology 
increases  significantly.  The 
basic  technological  components 
are  integrated  with  reasonably 
realistic  supporting  elements  so 
they  can  be  tested  in  a  simulated 
environment.  Examples  include 
“high-fidelity”  laboratory 
integration  of  components. 

Results  from  testing  a  laboratory 
breadboard  system  that  are 
integrated  with  other  supporting 
elements  in  a  simulated 
operational  environment.  How 
does  the  “relevant  environment” 
differ  from  the  expected 
operational  environment?  How 
do  the  test  results  compare  with 
expectations?  What  problems,  if 
any,  were  encountered?  Was 
the  breadboard  system  refined 
to  match  the  expected  system 
goals  more  nearly? 
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Table  MI-1.  TRL  Definitions,  Descriptions,  and  Supporting  Information 
(Source:  Interim  Guidebook,  dated  October  30,  2002)  (Continued) 


TRL 

Definition 

Description 

Supporting  Information 

6 

System/subsystem  model  or 
prototype  demonstration  in  a 
relevant  environment 

Representative  model  or 
prototype  system,  which  is  well 
beyond  that  of  TRL  5,  is  tested 
in  a  relevant  environment. 
Represents  a  major  step  up  in  a 
technology’s  demonstrated 
readiness.  Examples  include 
testing  a  prototype  in  a  high- 
fidelity  laboratory  environment 
or  in  [a]  simulated  operational 
environment. 

Results  from  laboratory  testing 
of  a  prototype  system  that  is 
near  the  desired  configuration 
in  terms  of  performance, 
weight,  and  volume.  How  did 
the  test  environment  differ  from 
the  operational  environment? 
Who  performed  the  tests?  How 
did  the  test  compare  with 
expectations?  What  problems, 
if  any,  were  encountered? 

What  are/were  the  plans, 
options,  or  actions  to  resolve 
problems  before  moving  to  the 
next  level? 

7 

System  prototype 
demonstration  in  an 
operational  environment 

Prototype  near,  or  at,  planned 
operational  system.  Represents 
a  major  step  up  from  TRL  6, 
requiring  demonstration  of  an 
actual  system  prototype  in  an 
operational  environment  such 
as  an  aircraft,  vehicle,  or  space. 
Examples  include  testing  the 
prototype  in  a  test  bed  aircraft. 

Results  from  testing  a 
prototype  system  in  an 
operational  environment.  Who 
performed  the  tests?  How  did 
the  test  compare  with 
expectations?  What  problems, 
if  any,  were  encountered? 

What  are/were  the  plans, 
options,  or  actions  to  resolve 
problems  before  moving  to  the 
next  level? 

8 

Actual  system  completed 
and  qualified  through  test 
and  demonstration 

Technology  has  been  proven  to 
work  in  its  final  form  and  under 
expected  conditions.  In  almost 
all  cases,  this  TRL  represents 
the  end  of  true  system 
development.  Examples  include 
developmental  test  and 
evaluation  of  the  system  in  its 
intended  weapon  system  to 
determine  if  it  meets  design 
specifications. 

Results  of  testing  the  system 
in  its  final  configuration  under 
the  expected  range  of 
environmental  conditions  in 
which  it  will  be  expected  to 
operate.  Assessment  of 
whether  it  will  meet  its 
operational  requirements.  What 
problems,  if  any,  were 
encountered?  What  are/were 
the  plans,  options,  or  actions 
to  resolve  problems  before 
finalizing  the  design? 

9 

Actual  system  proven 
through  successful  mission 
operations 

Actual  application  of  the 
technology  in  its  final  form  and 
under  mission  conditions,  such 
as  those  encountered  in 
operational  test  and  evaluation. 
Examples  include  using  the 
system  under  operational 
mission  conditions. 

Operational  test  and  evaluation 
reports. 
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Table  111-2.  Additional  Definitions  of  TRL  Descriptive  Terms 
(Source:  Interim  Guidebook ,  dated  October  30,  2002) 


Term 

Definition 

Breadboard 

Integrated  components  that  provide  a  representation  of  a 
system/subsystem  and  that  can  be  used  to  determine  concept 
feasibility  and  to  develop  technical  data.  Typically  configured  for 
laboratory  use  to  demonstrate  the  technical  principles  of 
immediate  interest.  May  resemble  final  system/subsystem  in 
function  only. 

High  Fidelity 

Addresses  form,  fit,  and  function.  High-fidelity  laboratory 
environment  would  involve  testing  with  equipment  that  can 
simulate  and  validate  all  system  specifications  within  a 
laboratory  setting. 

Low  Fidelity 

A  representative  of  the  component  or  system  that  has  limited 
ability  to  provide  anything  but  first-order  information  about  the 
end  product.  Low-fidelity  assessments  are  used  to  provide  trend 
analysis. 

Model 

A  functional  form  of  a  system,  generally  reduced  in  scale,  near  or 
at  operational  specification.  Models  will  be  sufficiently  hardened 
to  allow  demonstration  of  the  technical  and  operational 
capabilities  required  of  the  final  system. 

Operational  Environment 

Environment  that  addresses  all  the  operational  requirements  and 
specifications  required  of  the  final  system  to  include 
platform/packaging. 

Prototype 

A  physical  or  virtual  model  used  to  evaluate  the  technical  or 
manufacturing  feasibility  or  military  utility  of  a  particular 
technology  or  process,  concept,  end  item,  or  system. 

Relevant  Environment 

Testing  environment  that  simulates  the  key  aspects  of  the 
operational  environment. 

Simulated  Operational  Environment 

Either  (1 )  a  real  environment  that  can  simulate  all  of  the 
operational  requirements  and  specifications  required  of  the  final 
system  or  (2)  a  simulated  environment  that  allows  for  testing  of  a 
virtual  prototype;  used  in  either  case  to  determine  whether  a 
developmental  system  meets  the  operational  requirements  and 
specifications  of  the  final  system. 
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IV.  THE  TRA  PROCESS 


4.1  ACTION  SEQUENCE  FOR  A  TRA 

DoDI  5000.2  includes  a  description  of  activities  that  occur  before  Milestone  A. 
A  collaborative  effort  produces  an  ICD  that  describes  the  requisite  capabilities  and  time 
phased,  operational  goals.26  The  analyses  that  lead  to  the  ICD  identify  a  preferred  concept 
to  be  refined  before  a  Milestone  A  decision.  “The  MDA  designates  the  lead  DoD  Compo¬ 
nents)  to  refine  the  initial  concept  selected,  approves  the  AoA  plan,  and  establishes  a  date 
for  a  Milestone  A  review.”27 

Figure  IV- 1  graphically  portrays  the  steps  that  the  DUSD(S&T)  normally  antici¬ 
pates  in  the  assessment  of  technology  readiness  for  an  ACAT  I  or  IA  milestone  review. 
These  steps  are  derived  from  information  in  the  Interim  Guidebook ,  as  modified  by  DoDI 
5000.2.  However,  the  information  in  the  guidebook  is  not  mandatory,  so  the  steps  are 
merely  suggested. 

During  Concept  Refinement,  an  AoA  is  conducted  to  refine  the  selected  concept.28 
The  AoA  identifies  needed  technologies  that  are  not  yet  mature.  A  plan  for  maturing  these 
technologies  is  then  described  in  a  Technology  Development  Strategy  (TDS)  which  is 
approved  by  the  MDA  at  Milestone  A.  The  following  phase,  TD,  matures  the  technologies 
and  reduces  the  risk. 

Starting  during  TD,  the  steps29  for  a  Milestone  B  TRA  are  as  follows: 

A.  For  the  system,  the  PM  or  Project  Leader  conducts  a  risk  assessment  and 
develops  an  Acquisition  Program  Baseline  (APB)  and  a  Work  Breakdown 
Structure  (WBS). 


26  DoDI  5000.2,  paragraphs  3.2  and  3.4.  For  more  detail  see  CJCSM  3170.01,  Operation  of  the  Joint 
Capabilities  Integration  and  Development  System,  dated  24  June  2003,  Enclosure  D. 
thttp://www. teao.saic.com/jfcom/ier/documents/m3 1700 l.pdf).  A  sponsor  (e.g.,  a  Service)  prepares  the 
ICD.  It  is  approved  in  the  Joint  Staff. 

27  DoDI  5000.2  paragraph  3.5.2. 

28  DoDI  5000.2,  enclosure  6,  paragraph  E6.5,  specifies  that  responsibility  for  the  AoA  will  not  be 
assigned  to  the  PM.  The  AoA  is  directed  by  the  Director,  Program  Analysis  and  Evaluation  (PA&E)  in 
OSD. 

29  The  steps  that  follow  (A-J)  are  marked  accordingly  in  Figure  IV- 1. 
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Figure  IV-1.  Flow  Diagram  for  the  TRA  Process 
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B.  From  the  WBS,  the  risk  assessment,  and  functional  analysis,  the  PM  identifies 
those  technologies  that  are  not  already  fully  mature  but  that  are  critical  to  the 
accomplishment  of  goals  for  program  cost  and  schedule  and  for  system  pro- 
ducibility,  cost,  and  operational  effectiveness.  These  will  be  listed  as  critical 
technologies. 

To  support  the  TRA  required  before  an  upcoming  Milestone  B  or  Milestone  C, 
the  PM  prepares  a  list  of  the  critical  technologies  and  a  rationale  for  declaring 
these  technologies  to  be  critical.  Substantiating  information  normally  consists 
of  descriptions  of  the  status  of  components  or  subsystems,  the  testing  that  has 
been  accomplished,  and  the  results  of  this  testing.  Test  environments  and 
results  are  described  in  relation  to  the  functional  needs  of  the  system  concept. 
At  least  16  weeks  before  a  scheduled  Milestone  B  or  Milestone  C  (see  Fig¬ 
ure  II- 1),  the  list  of  critical  technologies  and  the  supporting  information  are  sent 
to  the  Component  S&T  Executive,  with  a  request  for  a  TRA.  At  the  same  time, 
an  information  copy  is  sent  to  the  DUSD(S&T). 

C.  The  Component  S&T  Executive  coordinates  with  the  PM  on  any  additions  to 
the  list  of  critical  technologies  and  on  any  additional  information  needed  for  the 
TRA. 

The  Component  S&T  Executive  directs  and  schedules  the  accomplishment  of  a 
TRA  based  on  the  PM’s  request  and  submission  of  the  critical  technologies 
information. 

The  TRA  is  conducted  in  accordance  with  Component  guidelines  and  proce¬ 
dures. 

D.  The  DUSD(S&T)  normally  appoints  a  member  of  his/her  staff  to  act  as  AO  to 
develop  a  basis  for  the  DUSD(S&T)  to  concur  with  the  Component  TRA.  This 
basis  must  be  sufficient  to  fulfill  the  DUSD(S&T)  oversight  responsibilities, 
but  it  should  not  be  a  duplication  of  the  Component  TRA. 

The  AO  should  review  the  critical  technologies  and  the  identification  process, 
negotiate  any  perceived  deficiencies,  and  provide  oversight  while  the  Compo¬ 
nent  TRA  is  conducted.  The  AO  should  coordinate  with  the  Component  S&T 
Executive  to  determine  to  what  extent  the  AO  or  technology  specialists  desig¬ 
nated  by  the  DUSD(S&T)  could  or  should  monitor  or  participate  in  the  Com¬ 
ponent  TRA.  The  Component  S&T  Executive  is  not  required  to  agree  to  any 
such  monitoring  or  participation  beyond  oversight. 

E.  When  the  Component  TRA  is  completed,  the  Component  S&T  Executive 
approves  it  and  forwards  it  to  the  CAE.  At  the  same  time,  the  Component  S&T 
Executive  sends  an  information  copy  to  the  DUSD(S&T). 

Subsequently,  the  CAE  forwards  the  approved  TRA  to  the  DUSD(S&T). 
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F.  The  AO  develops  a  basis  for  DUSD(S&T)  concurrence.  The  approach  can  be 
tailored  to  the  specific  situation  (see  paragraph  4.2,  which  describes  one 
approach).  The  AO  should  minimize  the  impact  on  the  PM  and  the  Component 
S&T  organization  but  still  provide  a  sound  basis  for  DUSD(S&T)  concur¬ 
rence.  Monitoring  or  participating  in  the  Component  TRA  will  likely  facilitate  a 
quick  concurrence.  If  the  AO  deems  any  critical  technology  to  be  insufficiently 
mature  for  the  coming  milestone,  he/she  tells  the  Component  S&T  Executive 
and  the  PM  so  that  all  involved  have  an  opportunity  to  reach  agreement  on 
appropriate  action. 

G.  Upon  receiving  the  report  and  official  TRA  from  the  CAE,  the  AO  confirms 
that  it  is  consistent  with  the  information  copy. 

H.  The  AO  prepares  a  memorandum  of  concurrence  or  nonconcurrence  for  sig¬ 
nature,  presents  the  staff  evaluation  of  the  TRA  to  the  DUSD(S&T),  provides 
whatever  backup  information  is  needed,  and  acts  on  the  DUSD(S&T)’s 
decision. 

I.  If  the  DUSD(S&T)  concurs,  the  concurrence  memorandum  is  transmitted  to 
the  OIPT  and  the  DAB  or  to  the  IT  OIPT  and  ITAB.  This  must  occur  at  least 
2  weeks  before  the  milestone  meeting. 

J.  If  the  DUSD(S&T)  does  not  concur,  an  independent  assessment  is  required. 
The  AO  recommends  a  course  of  action  and  prepares  a  memorandum  directing 
this  action.  The  independent  assessment  should  be  a  positive  contribution  to  the 
acquisition  program.  For  example,  it  could  result  in  a  revised,  more  realistic 
schedule,  in  the  use  of  an  alternative  technology,  or  in  a  revised,  evolutionary 
acquisition  strategy.  The  independent  assessment  should  be  conducted  as 
quickly  as  possible— whether  this  requires  1  day  or  several  months.  Typically, 
the  Component  funds  the  independent  assessment. 

This  process  (Figure  IV- 1)  applies  directly  to  Milestone  B.  It  can  also  be  applied  to 
Milestone  C  with  minor  changes.  Before  Milestone  C,  the  complete  system  is  demonstrated 
in  its  intended  environment.  This  demonstration  can  use  prototypes  or  EDMs.  If  the  proto¬ 
types  or  EDMs  are  complete  (for  the  critical  technologies),  all  the  critical  technologies  iden¬ 
tified  before  Milestone  B  that  are  used  in  the  system  design  (i.e.,  that  were  not  replaced  by 
other,  mature  technologies)  are  demonstrated.  The  TRA  conducted  in  preparation  for  Mile¬ 
stone  C,  then,  can  simply  document  the  use  of  these  technologies  and  show  that  the  system 
meets  the  requirements30  using  these  technologies. 


30  Requirements  are  documented  in  the  CDD  available  before  Milestone  B  and  in  the  CPD  available  before 
Milestone  C. 
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The  TRA  for  Milestone  C  should  also  include  any  technologies  that  were  found  to 
be  critical  during  SDD.  Critical  manufacturing  processes  are  identified  before  the  DRR,  and 
these  processes  could  involve  critical  technologies.  If  so,  these  critical  technologies  should 
be  demonstrated  before  Milestone  C  and  documented  in  the  TRA.  The  manufacturing  pro¬ 
cesses  themselves,  as  distinguished  from  the  technology  upon  which  they  depend,  are  not 
part  of  the  TRA. 

The  importance  of  manufacturing  and  producibility  to  system  development  and  pro¬ 
duction,  viewed  in  the  light  of  the  fundamentals  and  benefits  of  the  TRL  concept,  has  led  to 
the  idea  of  Manufacturing  Readiness  Levels,  or  MRLs.  Within  DoD,  a  Transition  Working 
Group,  comprised  of  representatives  from  the  Military  Services,  Defense  Logistics  Agency 
(DLA),  Missile  Defense  Agency  (MDA),  and  industry,  has  developed  an  initial  set  of  defi¬ 
nitions  and  descriptions  for  MRLs  that  are  suitable  for  consideration  and  use  in  defense 
acquisition.  Appendix  I  provides  these  MRL  definitions  and  descriptions.  Manufacturing 
readiness,  assessed  on  the  basis  of  MRLs,  is  not  currently  a  part  of  the  acquisition  system. 

Paragraph  4.2  offers  an  approach  to  developing  the  basis  for  DUSD(S&T) 
concurrence. 

4.2  DUSD(S&T)  CONCURRENCE 

The  DUSD(S&T)  is  required  to  evaluate  the  Component  TRA  before  Milestone  B 
and  Milestone  C  of  ACAT  ID  and  ACAT  IAM  programs.  An  AO,  designated  by  the 
DUSD(S&T),  will  normally  lead  the  evaluation  effort. 

It  is  recommended  that  the  AO  secure  DUSD(S&T)  concurrence  as  follows: 

•  When  the  DUSD(S&T)  designates  an  AO,  the  DUSD(S&T)  sends  a  memo¬ 
randum  to  the  Directors  of  his/her  staff.  This  memorandum  alerts  them  to  a 
possible  need  to  provide  assistance  in  their  respective  technology  areas  and 
requests  them  to  designate  a  point  of  contact  (POC)  within  their  Directorates.  If 
other  elements  of  the  USD(AT&L)  organization  are  needed  to  support  the 
TRA  evaluation,  that  support  is  also  requested  by  memorandum. 

•  The  AO  provides  copies  of  the  Component  TRA  to  the  designated  POCs  and 
invites  comments  by  a  certain  date. 

•  The  AO  reviews  the  TRA  and  calls  for  assistance,  as  necessary,  to  obtain  a 
competent  assessment  of  the  critical  technologies  or  to  determine  whether  all 
the  critical  technologies  have  been  identified. 

•  If  a  disagreement  with  the  Component  TRA  emerges,  this  is  noted  in  a  memo¬ 
randum  to  the  DUSD(S&T).  If  the  disagreement  would  jeopardize  a  favorable 
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decision  by  the  USD(AT&L)  or  the  ASD(NII),  the  AO  obtains  a  full  explana¬ 
tion  (and  concurrence  with  the  memorandum)  from  the  cognizant  Director. 

•  The  AO  conveys  the  evaluation  results  to  the  DUSD(S&T)  in  a  briefing  or 
memorandum.  Key  Directors  attend  or  coordinate. 

•  If  the  DUSD(S&T)  does  not  concur  with  the  Component  TRA,  the  AO  pre¬ 
pares  the  action  memorandum  to  conduct  an  independent  TRA. 

•  The  AO  prepares  a  memorandum  for  DUSD(S&T)  signature.  This  memoran¬ 
dum  gives  the  evaluation  results  of  the  Component  TRA  and  the  independent 
TRA,  if  conducted.  It  is  sent  to  the  Chairman  of  the  OIPT  or  IT  OIPT  and  to 
the  Executive  Secretary  of  the  DAB  or  the  appropriate  staff  officer  for  the 
ITAB. 
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V.  SUBMITTING  A  TRA 


5.1  SKELETAL  TEMPLATE  FOR  A  TRA  SUBMISSION 

The  following  outline  is  a  skeletal  template  for  anticipated  TRA  submissions: 

1.0  Purpose  of  This  Document 
2.0  Program  Overview 

2.1  Program  Objective 

2.2  Program  Description 

2.3  System  Description 

3.0  Technology  Readiness  Assessment 

3.1  Process  Description 

3.2  Critical  Technologies 

3.3  Assessment  of  Maturity 

3.3.1  First  Critical  Technology  or  Category  of  Technology 

3.3.2  Next  Critical  Technology  or  Category  of  Technology 

3.4  Summary  of  TRLs  by  Technology 

4.0  Conclusion 
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5.2  ANNOTATED  TEMPLATE  FOR  A  TRA  SUBMISSION 

The  following  outline  is  an  annotated  version  of  the  TRA  template.31 

1.0  Purpose  of  This  Document 

Should  be  short  and  should  give  the  program  name,  the  system  name  if  dif¬ 
ferent  from  the  program  name,  and  the  milestone  or  other  decision  point  for  which 
the  TRA  was  performed.  For  example,  “This  document  presents  an  independent 
Technology  Readiness  Assessment  (TRA)  for  the  UH-60M  helicopter  program  in 
support  of  the  Milestone  B  decision.  The  TRA  was  performed  at  the  direction  of  the 
Army  S&T  Executive.” 

2.0  Program  Overview 

2.1  Program  Objective 

States  what  the  program  is  trying  to  achieve  (e.g.,  new  capability,  improved 
capability,  lower  procurement  cost,  reduced  maintenance  or  manning,  and  so  forth). 
Refers  to  the  CDD  (for  Milestone  B)  or  the  Capability  Production  Document 
(CPD)  (for  Milestone  C)  that  documents  the  program  objectives. 

2.2  Program  Description 

Describes  the  program,  not  the  system.  Does  the  program  provide  a  new 
system  or  a  modification  to  an  existing  operational  system?  Is  it  an  evolutionary 
acquisition  program?  What  capabilities  will  be  realized  in  Block  1?  When  is  the  ini¬ 
tial  operational  capability  (IOC)?  Does  it  have  multiple  competing  prime  contrac¬ 
tors?  Into  what  architecture  does  it  fit?  Is  it  a  system-of-systems?  Does  its  success 
depend  on  the  success  of  other  acquisition  programs? 

2.3  System  Description 

Describes  the  overall  system,  the  major  subsystems,  and  components,  as 
necessary,  to  give  an  understanding  of  what  is  being  developed  and  to  show  what  is 
new,  unique,  or  special  about  it.  This  should  include  the  systems,  components,  and 


31  Appendix  F  contains  two  examples  of  TRA  submissions  to  OSD. 
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technologies  that  will  later  be  declared  “critical  technologies.”  Describes  how  the 
system  works  (if  this  is  not  obvious). 

3.0  Technology  Readiness  Assessment 

3.1  Process  Description 

Tells  who  led  the  TRA  and  what  organizations  or  individuals  performed  the 
TRA.  Identifies  the  special  expertise  of  participating  organizations  or  individuals. 
This  should  establish  the  competence  and  the  independence  of  the  TRA.  In  this 
context,  “independence”  means  that  the  assessors  are  not  unduly  influenced  by  the 
opinions  of  the  developers  (government  or  industry).  Usually,  the  PM  or  the  System 
Program  Office  (SPO)  will  provide  most  of  the  data  and  other  information  that  form 
the  basis  of  a  TRA.  Nevertheless,  the  assessment  should  be  independent  of  the  PM 
or  SPO. 

States  what  analyses  and  investigations  were  performed  when  making  the 
assessment  (e.g.,  examination  of  test  setups,  discussions  with  test  personnel,  analy¬ 
sis  of  test  data,  review  of  related  technology,  and  so  forth). 

This  is  only  a  broad  description  of  the  process.  Paragraph  3.3  presents  an 
opportunity  to  include  more  detail. 

3.2  Critical  Technologies 

Lists  the  technologies  included  in  the  TRA.  A  table  that  lists  the  technology 
name  and  includes  a  few  words  that  describe  the  technology  and  its  function  is 
appropriate.  The  technologies  can  be  organized  according  to  the  WBS,  as  provided 
by  the  PM.  The  names  of  these  critical  technologies  should  be  used  consistently 
throughout  the  remainder  of  the  document. 

The  PM  should  identify  the  critical  technologies.  The  Component  S&T 
Executive  should  assess  at  least  these  technologies;  however,  other  technologies  that 
the  Component  S&T  Executive  considers  critical  can  also  be  included. 
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3.3  Assessment  of  Maturity 


3.3.1  First  Critical  Technology  or  Category  of  Technology 

Describes  the  technology  (subsystem,  component,  or  technology).  Describes 
the  function  it  performs  and,  if  needed,  how  it  relates  to  other  parts  of  the  system. 
Provides  a  synopsis  of  TD  history  and  status.  This  can  include  facts  about  related 
uses  of  the  same  or  similar  technology,  numbers  or  hours  of  testing  of  breadboards, 
numbers  of  prototypes  built  and  tested,  relevance  of  the  test  conditions,  and  results 
achieved.  Finally,  applies  the  criteria  for  TRLs  and  assigns  a  readiness  level  to  the 
technology.  States  the  readiness  level  (e.g.,  TRL  5)  and  the  rationale  for  choosing 
this  readiness  level. 

For  a  complex  system,  if  the  critical  technologies  presented  are  in  categories 
(e.g.,  airframe  or  sensors),  the  information  specified  in  the  previous  paragraph  (e.g., 
describing  the  technology,  describing  the  function  it  performs,  and  so  forth)  should 
be  provided  for  each  critical  technology  within  a  category. 

3.3.2  Next  Critical  Technology  or  Category  of  Technology 

This  paragraph  and  the  following  paragraphs  (e.g.,  3.3.3,  3.3.4,  and  so  forth) 
present  for  other  critical  technologies  the  same  type  of  information  that  was 
presented  in  paragraph  3.3.1. 

3.4  Summary  of  TRLs  by  Technology 

Presents  a  table  that  lists  the  critical  technologies  and,  for  each  critical  tech¬ 
nology,  presents  the  TRL  assigned  and  a  short  explanation  (one  sentence  or  a  list  of 
factors). 

4.0  Conclusion 

States  the  Component  S&T  Executive’s  position  concerning  the  maturity  of 
the  technologies  and  whether  this  maturity  is  adequate  for  the  system  to  enter  the 
next  stage  of  development.  If  the  position  is  supportive  of  entering  the  next  stage 
even  though  some  critical  technologies  are  less  mature  than  would  ordinarily  be 
expected,  explains  what  circumstances  or  planned  work  justifies  the  positive 
position. 
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The  TRA  should  be  signed  “Approved  By”  the  Component  S&T 
Executive,  or  it  should  be  transmitted  with  a  cover  memorandum  that  clearly  states 
that  the  TRA  represents  the  position  of  the  Component  S&T  Executive.  In  effect,  the 
Component  S&T  Executive  must  certify  that  he/she  stands  behind  the  statements  in 
the  Conclusion. 
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ACAT 

Acquisition  Category 

AIS 

Automated  Information  System 

AO 

Action  Officer 

AoA 

Analysis  of  Alternatives 

APB 

Acquisition  Program  Baseline 

ASD(C3I) 

Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications,  and  Intelligence 

ASD(NII) 

Assistant  Secretary  of  Defense  for  Networks  and  Information 
Integration 

CAE 

Component  Acquisition  Executive 

CDD 

Capability  Development  Document 

CJCS 

Chairman  of  the  Joint  Chiefs  of  Staff 

CJCSM 

Chairman  of  the  Joint  Chiefs  of  Staff  Manual 

CPD 

Capability  Production  Document 

DAB 

Defense  Acquisition  Board 

DAS 

Defense  Acquisition  System 

DDR&E 

Director  of  Defense  Research  and  Engineering 

DLA 

Defense  Logistics  Agency 

DoD  CIO 

DoD  Chief  Information  Officer 

DoD 

Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

DRR 

Design  Readiness  Review 

DUSD(S&T) 

Deputy  Under  Secretary  of  Defense  for  Science  and  Technology 

EDM 

engineering  development  model 

FOC 

full  operational  capability 

FRP 

full-rate  production 

GAO 

Government  Accounting  Office 

ICD 

Initial  Capabilities  Document 

IOC 

initial  operational  capability 

IOT&E 

Initial  Operational  Test  and  Evaluation 
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IT  OIPT 

Information  Technology  Overarching  Integrated  Product  Team 

ITAB 

Information  Technology  Acquisition  Board 

JROC 

Joint  Requirements  Oversight  Council 

KPP 

key  performance  parameter 

LRIP 

low  rate  initial  production 

MAIS 

Major  Automated  Information  System 

MDA 

Milestone  Decision  Authority 

Missile  Defense  Agency 

MDAP 

Major  Defense  Acquisition  Program 

MRL 

Manufacturing  Readiness  Level 

NASA 

National  Aeronautics  and  Space  Administration 

NDAA 

National  Defense  Authorization  Act 

ODDR&E 

Office  of  the  Director  of  Defense  Research  and  Engineering 

OIPT 

Overarching  Integrated  Product  Team 

OSD 

Office  of  the  Secretary  of  Defense 

PA&E 

Program  Analysis  and  Evaluation 

PEO 

Program  Executive  Officer 

PM 

Program  Manager 

POC 

point  of  contact 

RDT&E 

research,  development,  test,  and  evaluation 

S&T 

Science  and  Technology 

SDD 

System  Development  and  Demonstration,  a  phase  in  the  DAS 

SPO 

System  Program  Office 

T&E 

test  and  evaluation 

TD 

Technology  Development 

TDS 

Technology  Development  Strategy 

TRA 

Technology  Readiness  Assessment 

TRL 

Technology  Readiness  Level 

USD(AT&L) 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics 

VCJCS 

Vice  Chairman  of  the  Joint  Chiefs  of  Staff 

WBS 

Work  Breakdown  Structure 
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Several  GAO  reports  addressed  the  DoD  acquisition  system  and  made  recommen¬ 
dations  that  influenced  the  DoD  5000  series  of  publications.  The  following  presents  a  brief 
summary  of  GAO-related  work,  along  with  references  for  the  source  documents. 

A.l  GAO  REPORTS 

The  subcommittee  on  Readiness  and  Management  Support  of  the  Committee  on 
Armed  Services,  U.S.  Senate,  which  has  oversight  on  acquisitions  policy,  enlisted  the  GAO 
in  a  study  of  best  commercial  practices  as  related  to  defense  acquisition.  A  series  of  GAO 
reports  and  related  testimony  assessed  how  best  commercial  practices  could  improve  the 
way  DoD  incorporates  new  technology  into  weapon  system  programs  and  reduces  risk. 
These  reports,  issued  from  1996-2000  (the  principal  of  which  are  listed  as  Refs.  1,  2,  3), 
offered  DoD  some  guidance  and  had  significant  influence  on  the  current  versions  of  the 
DoD  5000  series  of  documents  [Department  of  Defense  Directive  (DoDD)  5000.1, 
Department  of  Defense  Instruction  (DoDI)  5000.2,  and  the  Interim  Defense  Acquisition 
Guidebook  (formerly  DoD  5000. 2-R)]  (Refs.  4,  5,  6). 

The  weapon  system  acquisition  cycle  for  DoD  major  weapon  systems  before  the 
issuance  of  References  4,  5,  and  6  could  be  illustrated  as  shown  in  Figure  A-l.  Technology, 
design,  and  manufacturing  knowledge  was  obtained  concurrently. 
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Figure  A-1.  DoD’s  Current  Weapon  System  Acquisition  Cycle 

The  major  GAO  recommendation  that  followed  best  commercial  practice  is  to  mini¬ 
mize  technology  development  during  product  development  and  match  requirements  with 
technological  capability  before  product  development  is  launched.  Proof  that  the  technology 
will  work  and  can  be  demonstrated  to  a  high  level  of  maturity  is  critical  to  lowering  risk  and 
avoiding  large  cost  overruns.  Associated  with  this  principle  are  the  needs  to  develop  high 
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standards  for  finding  the  maturity  and  readiness  of  technology,  to  establish  disciplined 
paths  that  technology  must  take  to  be  included  in  products,  and  to  provide  strong  gate¬ 
keepers  to  decide  when  to  allow  the  technology  into  a  product  development  program.  GAO 
recommended  that  DoD  not  launch  a  program  until  the  technologies  needed  to  meet  a  new 
weapons  requirement  are  mature.  To  separate  this  technology  development  from  the  pro¬ 
gram,  GAO  best  practices  recommendations  suggest  that  a  technology  and  concept  matura¬ 
tion  phase  follow  concept  exploration  and  precede  program  launch,  as  illustrated  in 
Figure  A-2. 
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Figure  A-2.  Weapon  Acquisition  Phases  That  Should  Precede 
the  Launch  of  a  New  Program 

The  GAO  review  of  best  practices  for  including  new  technology  in  products  (see 
Ref.  2)  applied  a  scale  of  Technology  Readiness  Levels  (TRLs)  pioneered  by  the  National 
Aeronautics  and  Space  Administration  (NASA)  and  adapted  by  the  Air  Force  Research 
Laboratory  (AFRL).  “TRLs  proved  to  be  reliable  indicators  of  the  relative  maturity  of  the 
23  technologies  reviewed,  both  commercial  and  military,  and  their  eventual  success  after 
they  were  included  in  product  development  programs”  (Ref.  2,  p.  22) 

To  show  that  design  is  mature,  the  GAO  studies  suggest  that  a  product  development 
phase  should  include  a  distinct  system  integration  effort  before  the  system  demonstration 
effort  to  demonstrate  the  effectiveness  of  the  product  and  processes.  See  Figure  A-3. 
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Figure  A-3.  Product  Development  Phase  To  Deliver  a 
Mature  Design  and  Key  Processes 
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Figure  A-4  shows  GAO’s  final  proposal  for  a  potential  DoD  technology  and  pro¬ 
duct  development  process  based  on  commercial  best  practices.  It  should  be  noted  that 
leading  commercial  firms  launch  a  new  product  later  than  DoD  — after  technology  is  com¬ 
plete.  Paragraphs  A.2  and  A.3  of  this  appendix  provide  the  GAO  recommendations  for 
DoD  management  of  Technology  Development  and  the  DoD  response  as  reported  in  Ref¬ 
erence  2.  DoD  did  not  agree  entirely  with  GAO’s  recommendations  and  is  willing  to  accept 
more  risk.  DoD  considered  TRL  6  as  an  acceptable  readiness-level  risk  for  a  weapon  sys¬ 
tem  entering  the  program  definition  stage  (see  Figure  A-l)  and  TRL  7  as  an  acceptable 
readiness-level  risk  for  the  Engineering  and  Manufacturing  Development  (EMD)  stage. 
GAO  accepted  this. 
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Figure  A-4.  Potential  DoD  Technology  and  Product  Development  Process 

Incorporating  Best  Practices 

Figure  A-5  outlines  the  current  Defense  Acquisition  Management  Framework  pre¬ 
sented  in  DoD  5000.2,  dated  May  12,  2003.  The  relationship  to  the  GAO  recommendation 
of  Figure  A-4  is  evident. 
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Figure  A-5.  Defense  Acquisition  Management  Framework 
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A.2  GAO  RECOMMENDATIONS 


The  following  paragraphs  are  direct  quotations  from  Reference  2:  GAO/NSIAD-99 
162,  Best  Practices:  Better  Management  of  Technology  Development  Can  Improve  Wea 
pon  System  Outcomes. 

We  have  previously  recommended  that  DOD  separate  technology  develop¬ 
ment  from  weapon  system  programs.  That  recommendation  was  made  with¬ 
out  prejudice  toward  the  necessity  of  technology  development  but  rather  with 
the  intent  that  programs  could  be  better  managed  if  such  development  was 
conducted  outside  of  a  program  manager’s  purview.  Similarly,  the  recom¬ 
mendations  that  follow  are  made  without  prejudice  toward-or  the  intention  of 
compromising-the  basic  research  and  other  activities  that  S&T  organizations 
perform.  We  recognize  that  implementation  of  these  recommendations  will 
have  organizational,  funding,  and  process  implications  and  will  require  the 
cooperation  of  the  Congress  (p.  62). 

To  help  ensure  that  new  technologies  are  vigorously  pursued  and  success¬ 
fully  moved  into  weapon  system  programs,  we  recommend  that  the  Secretary 
of  Defense  adopt  a  disciplined  and  knowledge-based  method  for  assessing 
technology  maturity,  such  as  TRLs,  DOD-wide.  This  practice  should 
employ  standards  for  assessing  risks  of  handoff  to  program  managers  that 
are  based  on  a  technology’s  level  of  demonstration  and  its  criticality  to 
meeting  the  weapon  system’s  requirements  (p.63). 

With  these  tools  in  hand,  we  recommend  that  the  Secretary  (1)  establish  the 
place  at  which  a  match  is  achieved  between  key  technologies  and  weapon 
system  requirements  as  the  proper  time  for  committing  to  the  cost,  schedule, 
and  performance  baseline  for  developing  and  producing  that  weapon  system 
and  (2)  require  that  key  technologies  reach  a  high  maturity  level— analogous 
to  TRL  7-before  making  that  commitment.  This  would  approximate  the 
launch  point  for  product  development  as  practiced  by  leading  commercial 
firms  (p.  63). 

We  recommend  that  the  Secretary  find  ways  to  ensure  that  the  managers 
responsible  for  maturing  the  technologies  and  designing  weapon  systems 
before  product  development  are  provided  the  more  flexible  environment  that 
is  suitable  for  the  discovery  of  knowledge,  as  distinct  from  the  delivery  of  a 
product.  Providing  more  flexibility  will  require  the  cooperation  of  require¬ 
ments  managers  and  resource  managers  so  that  rigid  requirements  or  the 
threat  of  jeopardizing  the  funding  planned  to  start  product  development  will 
not  put  pressure  on  program  managers  to  accept  immature  technologies. 

Such  an  environment  may  not  be  feasible  if  the  program  definition  and  risk 
reduction  phase  remains  the  effective  launch  point  for  an  entire  weapon  sys¬ 
tem  program  (p.  63). 

An  implication  of  these  recommendations  is  that  S&T  organizations  will 
have  to  play  a  greater  role  in  maturing  technologies  to  higher  levels  and 
should  be  funded  accordingly.  Therefore,  we  recommend  that  the  Secretary 
of  Defense  evaluate  the  different  ways  S&T  organizations  can  play  a  greater 
role  in  helping  technologies  reach  high  levels  of  maturity  before  product 
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development  begins.  For  example,  given  that  a  technology  has  sufficient 
potential  for  application  to  a  weapon  system,  at  a  minimum,  an  S&T  organi¬ 
zation  should  be  responsible  for  taking  a  technology  to  TRL  6  before  it  is 
handed  off  to  a  program  office  at  the  program  definition  and  risk  reduction 
phase.  During  this  phase,  the  program  manager  would  be  responsible  for 
maturing  the  technology  to  TRL  7  before  it  is  included  in  an  engineering  and 
manufacturing  development  program.  In  a  situation  where  a  single,  design¬ 
pacing  technology  is  to  be  developed  for  a  known  application— like  the  non¬ 
penetrating  periscope— an  S&T  organization  should  be  required  to  mature 
that  technology  to  TRL  7  before  it  is  turned  over  to  a  product  development 
manager.  S&T  organizations  could  play  a  similar  role  when  a  significant 
new  technology  is  being  prepared  for  insertion  into  an  existing  weapon  sys¬ 
tem.  Finally,  when  multiple  new  technologies  are  to  be  merged  to  create  a 
weapon  system,  S&T  organizations  should  be  required  to  bring  key  tech¬ 
nologies  to  TRL  6  and  then  become  part  of  a  hybrid  organization  with  prod¬ 
uct  developers  to  integrate  the  technologies  and  bring  them  to  TRL  7  before 
handing  full  responsibility  to  a  product  development  manager  (pp.  63-64). 

To  help  guard  against  the  possibility  that  the  more  basic  research  and  tech¬ 
nology  development  activities  would  be  compromised  by  having  S&T  orga¬ 
nizations  routinely  take  key  technologies  to  TRL  6  or  higher,  we  recommend 
that  the  Secretary  extract  lessons  from  the  nonpenetrating  periscope,  the 
AAAV,  and  the  Army’s  Future  Scout  programs,  and  other  ATD  and  ACTD 
programs.  Specifically,  the  Secretary  should  assess  whether  the  resources 
needed  to  enable  S&T  organizations  to  play  a  leading  role  in  the  develop¬ 
ment  of  technologies  and,  in  some  cases,  preliminary  system  design, 
detracted  from  or  displaced  more  basic  research  and  technology  develop¬ 
ment  programs  (p.  64). 

Finally,  we  recommend  that  the  Secretary  empower  managers  of  product 
development  programs  to  refuse  to  accept  key  technologies  with  low  levels 
of  demonstrated  maturity.  The  Secretary  can  encourage  this  behavior 
through  supportive  decisions  on  individual  programs,  such  as  by  denying 
proposals  to  defer  the  development  of  key  technologies  and  by  favoring 
proposals  to  lengthen  schedules  or  lessen  requirements  to  reduce  techno¬ 
logical  risk  early  (p.  64). 

A.3  DoD  COMMENTS  AND  GAO  EVALUATION 

The  following  paragraphs  are  direct  quotations  from  Reference  2:  GAO/NSIAD-99- 

162,  Best  Practices:  Better  Management  of  Technology  Development  Can  Improve 

Weapon  System  Outcomes. 

DOD  generally  concurred  with  a  draft  of  this  report  and  its  recommenda¬ 
tions,  noting  that  the  traditional  path  to  new  weapon  system  development  is 
no  longer  affordable  or  necessary  (see  app.  I).  DOD  stated  that  it  has 
embarked  upon  a  “Revolution  in  Business  Affairs”  that  will  enable  new 
technologies  to  be  developed  more  efficiently  and  effectively.  It  believes  that 
the  first  steps  in  this  direction  have  already  been  taken  but  agrees  that  more 
progress  needs  to  be  made.  DOD  agreed  that  TRLs  are  necessary  in 
assisting  decision-makers  in  deciding  on  when  and  where  to  insert  new 
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technologies  into  weapon  system  programs  and  that  weapon  system  manag¬ 
ers  should  ensure  that  technology  is  matured  to  a  TRL  7  before  insertion 
occurs.  DOD  concurred  that  S&T  organizations  should  be  involved  in 
maturing  technologies  to  high  levels,  such  as  TRL  6,  before  transitioning  to 
the  engineering  and  manufacturing  development  phase  and  agreed  to  assess 
the  impact  of  this  involvement  on  other  S&T  resources.  We  note  that  the 
best  practice  is  to  mature  technology  to  at  least  a  TRL  7  before  starting  the 
engineering  and  manufacturing  development  phase,  whether  the  technology 
is  managed  by  an  S&T  organization,  a  weapon  system  program  manager,  or 
a  hybrid  of  the  two  organizations  (pp.  64—65). 

DOD  noted  that  while  TRLs  are  important  and  necessary,  the  increasing 
projected  life  for  new  weapon  systems,  total  ownership  costs,  and  urgency 
based  upon  threat  assessments  are  also  important  considerations  for  system 
development  decisions.  We  agree  and  note  that  our  recommendations  are  not 
intended  to  cover  all  aspects  of  weapon  system  development  decisions  or  to 
suggest  that  technology  maturity  is  the  only  factor  in  such  decisions.  Rather, 
the  recommendations  are  in  keeping  with  the  purpose  of  the  report,  “to 
determine  whether  best  practices  offer  methods  to  improve  the  way  DOD 
matures  new  technology  so  that  it  can  be  assimilated  into  weapon  system 
programs  with  less  disruption.”  We  believe  that  a  knowledge-based 
approach  to  maturing  technology,  such  as  TRLs,  can  benefit  other  consid¬ 
erations  as  well.  For  example,  decisions  on  what  technologies  to  include  in  a 
weapon  system  and  when  to  include  them  can  have  a  significant  bearing  on 
its  total  ownership  costs. 

DOD  stated  that  there  should  be  an  established  point  for  the  transition  of 
technologies  and  that  it  plans  to  supplement  its  milestone  review  process 
with  additional  guidance  in  the  next  revisions  to  DOD  5000. 2-R.  It  also 
stated  that  its  policy  on  the  evolutionary  approach  to  weapon  acquisitions 
should  be  developed  in  consonance  with  the  technology  transition  strategy. 
We  cannot  comment  on  the  revisions  to  the  directive  or  the  evolutionary 
acquisition  policy  because  they  have  yet  to  be  published.  However,  under  the 
current  milestone  review  process,  the  pressures  placed  on  a  program  during 
the  program  definition  and  risk  reduction  phase— when  much  technology 
development  occurs— can  operate  against  the  flexibility  and  judgments  that 
are  needed  to  mature  technologies.  If  the  revisions  to  the  directive  supple¬ 
ment  the  current  milestones  without  relieving  the  pressures  brought  to  bear 
on  programs  as  they  are  launched  in  the  program  definition  and  risk  reduc¬ 
tion  phase,  it  will  remain  difficult  to  discourage  the  acceptance  of  immature 
technologies  in  the  design  of  new  weapon  systems.  To  relieve  these  pres¬ 
sures,  we  encourage  DOD,  as  it  develops  the  directive  and  the  evolutionary 
acquisition  policy,  to  separate  technology  development  from  product  devel¬ 
opment  and  to  redefine  the  launch  point  for  a  program  as  the  point  at  which 
enough  knowledge  has  been  gained  to  ensure  that  a  match  is  reached 
between  the  maturity  of  key  technologies  and  weapon  system  requirements 
(pp.  65-66). 

DOD  also  stated  that  program  managers  already  have  the  ability  to  reject 
inappropriately  mature  technologies,  and  to  the  extent  technology  immaturity 
affects  acquisition  baselines,  to  advise  acquisition  executives  of  feasible 
alternatives.  We  did  not  find  this  to  be  the  case  in  our  review.  Rather,  we 
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found  that  the  program  managers’  ability  to  reject  immature  technologies  is 
hampered  by  (1)  untradable  requirements  that  force  acceptance  of  technolo¬ 
gies  despite  then-  immaturity  and  (2)  reliance  on  tools  for  judging  technol¬ 
ogy  maturity  that  fail  to  alert  the  managers  of  the  high  risks  that  would 
prompt  such  a  rejection.  As  noted  in  the  report,  once  a  weapon  system  pro¬ 
gram  begins,  the  environment  becomes  inflexible  and  deviations  to  program 
baselines  can  attract  unwanted  attention.  This  reality  limits  the  program  man¬ 
agers’  ability  to  reject  immature  technologies  (p.  66). 
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ACRONYMS  AND  ABBREVIATIONS  FOR  APPENDIX  A 


AAAV 

Advanced  Amphibious  Assault  Vehicle 

ACAT 

Acquisition  Category 

ACTD 

Advanced  Concept  Technology  Demonstration 

AFRL 

Air  Force  Research  Laboratory 

ASD(C3I) 

Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications,  and  Intelligence 

ATD 

Advanced  Technology  Demonstration 

CAE 

Component  Acquisition  Executive 

DAB 

Defense  Acquisition  Board 

DoD 

Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

DUSD(S&T) 

Deputy  Under  Secretary  of  Defense  for  Science  and 
Technology 

EMD 

Engineering  and  Manufacturing  Development 

FOC 

full  operational  capability 

GAO 

General  Accounting  Office 

IOC 

initial  operational  capability 

LRIP 

low  rate  initial  production 

MAIS 

Major  Automated  Information  System 

MDAP 

Major  Defense  Acquisition  Program 

MNS 

Mission  Needs  Statement 

NASA 

National  Aeronautics  and  Space  Administration 

NSIAD 

National  Security  and  International  Affairs  Division  (GAO) 

ODDR&E 

Office  of  the  Director  of  Defense  Research  and  Engineering 

ODUSD(S&T) 

Office  of  the  Deputy  Under  Secretary  of  Defense  for 
Science  and  Technology 

ORD 

Operational  Requirements  Document 

PM 

Program  Manager 

S&T 

Science  and  Technology 

TRA 

Technology  Readiness  Assessment 

TRL 

Technology  Readiness  Level 

USD(AT&L) 

Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics 
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The  DoD  5000  series  documents  relevant  to  TRAs  are 

•  Department  of  Defense  Directive  (DoDD)  5000.1,  The  Defense  Acquisition 
System,  dated  May  12,  2003. 

•  Department  of  Defense  Instruction  (DoDI)  5000.2,  Operation  of  the  Defense 
Acquisition  System,  dated  May  12,  2003. 

•  Interim  Defense  Acquisition  Guidebook,  dated  October  30,  2002,  [formerly 
DoD  5000.2-R,  Mandatory  Procedures  for  Major  Defense  Acquisition  Pro¬ 
grams  (MDAPs)  and  Major  Automated  Information  System  (MAIS)  Acquisi¬ 
tion  Programs,  dated  April  5,  2002)]. 

For  background  and  reference,  portions  of  these  documents  relevant  to  Technology 
Readiness.  These  DoD  5000  series  documents  appear  on  Internet  Web  site 
http://dod5000.dau.miF. 

B.l  EXTRACTS  FROM  DoD  5000  SERIES  DOCUMENTS  RELEVANT  TO 
TECHNOLOGY  READINESS  ASSESSMENT 

B.1.1  DoDD  5000.1,  dated  May  12,  2003 

•  Policy 

4.3.  The  following  policies  shall  govern  the  Defense  Acquisition  System: 

4.3.1.  Flexibility.  There  is  no  one  best  way  to  structure  an  acquisition  program 
to  accomplish  the  objective  of  the  Defense  Acquisition  System.  MDAs  and 
PMs  shall  tailor  program  strategies  and  oversight,  including  documentation  of 
program  information,  acquisition  phases,  the  timing  and  scope  of  decision 
reviews,  and  decision  levels,  to  fit  the  particular  conditions  of  that  program, 
consistent  with  applicable  laws  and  regulations  and  the  time-sensitivity  of  the 
capability  need. 

4.3.2.  Responsiveness.  Advanced  technology  shall  be  integrated  into  producible 
systems  and  deployed  in  the  shortest  time  practicable.  Approved,  time-phased 
capability  needs  matched  with  available  technology  and  resources  enable  evolu¬ 
tionary  acquisition  strategies.  Evolutionary  acquisition  strategies  are  the 
preferred  approach  to  satisfying  operational  needs.  Spiral  development  is  the 
preferred  process  for  executing  such  strategies. 

•  Enclosure  1:  Additional  Policy 

El. 14.  Knowledge-Based  Acquisition.  PMs  shall  provide  knowledge  about  key 
aspects  of  a  system  at  key  points  in  the  acquisition  process.  PMs  shall  reduce 
technology  risk,  demonstrate  technologies  in  a  relevant  environment,  and  iden¬ 
tify  technology  alternatives,  prior  to  program  initiation.  They  shall  reduce 
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integration  risk  and  demonstrate  product  design  prior  to  the  design  readiness 
review.  They  shall  reduce  manufacturing  risk  and  demonstrate  producibility 
prior  to  full-rate  production. 

E1.28.  Technology  Development  and  Transition.  The  Science  and  Technology 
(S&T)  program  shall: 

E.l.28.1.  Address  user  needs; 

E.l.28.2.  Maintain  a  broad-based  program  spanning  all  Defense-relevant  sci¬ 
ences  and  technologies  to  anticipate  future  needs  and  those  not  being  pursued 
by  civil  or  commercial  communities; 

El.28.3.  Preserve  long-range  research;  and 

E.l.28.4.  Enable  rapid,  successful  transition  from  the  S&T  base  to  useful  mili¬ 
tary  products. 

B.1.2  DoDI  5000.2,  dated  May  12,  2003 

•  Applicability  and  Scope 

2.2.  All  defense  technology  projects  and  acquisition  programs.  Some  require¬ 
ments,  where  stated,  apply  only  to  Major  Defense  Acquisition  Programs 
(MDAPs)  and  Major  Automated  Information  System  (MAIS)  programs. 

•  User  Needs  and  Technology  Opportunities 

3.4.1.  The  capability  needs  and  acquisition  management  systems  shall  use  Joint 
Concepts,  integrated  architectures,  and  an  analysis  of  doctrine,  organization, 
training,  materiel,  leadership,  personnel,  and  facilities  (DOTMLPF)  in  an  inte¬ 
grated,  collaborative  process  to  define  desired  capabilities  to  guide  the  devel¬ 
opment  of  affordable  systems.  The  Chairman  of  the  Joint  Chiefs  of  Staff,  with 
the  assistance  of  the  Joint  Requirements  Oversight  Council,  shall  assess  and 
provide  advice  regarding  military  capability  needs  for  defense  acquisition 
programs.  The  process  through  which  the  Chairman  provides  his  advice  is  des¬ 
cribed  in  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170.01  (refer¬ 
ence  (g)).  Representatives  from  multiple  DoD  communities  shall  assist  in 
formulating  broad,  time -phased,  operational  goals,  and  describing  requisite 
capabilities  in  the  Initial  Capabilities  Document  (ICD).  They  shall  examine 
multiple  concepts  and  materiel  approaches  to  optimize  the  way  the  Department 
of  Defense  provides  these  capabilities.  The  examination  shall  include  robust 
analyses  that  consider  affordability,  technology  maturity,  and  responsiveness. 

Concept  Refinement 

3.5.2.  Concept  Refinement  begins  with  the  Concept  Decision.  The  MDA  des¬ 
ignates  the  lead  DoD  Component(s)  to  refine  the  initial  concept  selected, 
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approves  the  AoA  plan,  and  establishes  a  date  for  a  Milestone  A  review.  The 
MDA  decisions  shall  be  documented  in  an  Acquisition  Decision  Memorandum 
(ADM).  This  effort  shall  normally  be  funded  only  for  the  concept  refinement 
work.  The  MDA  decision  to  begin  Concept  Refinement  DOES  NOT  mean  that 
a  new  acquisition  program  has  been  initiated.  The  tables  in  enclosure  3  identify 
all  statutory  and  regulatory  requirements  for  the  Concept  Refinement  decision. 

3.5.3.  The  ICD  and  the  AoA  plan  shall  guide  Concept  Refinement.  The  focus 
of  the  AoA  is  to  refine  the  selected  concept  documented  in  the  approved  ICD. 
The  AoA  shall  assess  the  critical  technologies  associated  with  these  concepts, 
including  technology  maturity,  technical  risk,  and,  if  necessary,  technology 
maturation  and  demonstration  needs.  To  achieve  the  best  possible  system  solu¬ 
tion,  emphasis  shall  be  placed  on  innovation  and  competition.  Existing  com¬ 
mercial-off-the-shelf  (COTS)  functionality  and  solutions  drawn  from  a 
diversified  range  of  large  and  small  businesses  shall  be  considered. 

•  Technology  Development 

3.6.1.  Purpose.  The  purpose  of  this  phase  is  to  reduce  technology  risk  and  to 
determine  the  appropriate  set  of  technologies  to  be  integrated  into  a  full  system. 
Technology  Development  is  a  continuous  technology  discovery  and  develop¬ 
ment  process  reflecting  close  collaboration  between  the  S&T  community,  the 
user,  and  the  system  developer.  It  is  an  iterative  process  designed  to  assess  the 
viability  of  technologies  while  simultaneously  refining  user  requirements. 

3.6.2.  The  project  shall  enter  Technology  Development  at  Milestone  A  when 
the  MDA  has  approved  the  TDS.  The  tables  in  enclosure  3  identify  all  statutory 
and  regulatory  requirements  applicable  to  Milestone  A.  This  effort  normally 
shall  be  funded  only  for  the  advanced  development  work.  For  business  area 
capabilities,  commercially  available  solutions  shall  be  employed.  (A  toolkit  of 
best  practices  is  available  at  http://deskbook.dau.mil).  A  favorable  Milestone  A 
decision  DOES  NOT  mean  that  a  new  acquisition  program  has  been  initiated. 

3.6.3.  Shipbuilding  programs  may  be  initiated  at  the  beginning  of  Technology 
Development.  The  information  required  in  the  tables  at  enclosure  3  shall  sup¬ 
port  program  initiation.  A  cost  assessment  shall  be  prepared  in  lieu  of  an  inde¬ 
pendent  cost  estimate  (ICE),  and  a  preliminary  assessment  of  the  maturity  of 
key  technologies  shall  be  provided. 

•  System  Development  and  Demonstration 
3.7.1.  Purpose 

3.7.I.2.  SDD  has  two  major  efforts:  System  Integration  and  System  Demon¬ 
stration.  The  entrance  point  is  Milestone  B,  which  is  also  the  initiation  of  an 
acquisition  program.  There  shall  be  only  one  Milestone  B  per  program  or  evo¬ 
lutionary  increment.  Each  increment  of  an  evolutionary  acquisition  shall  have 
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its  own  Milestone  B.  The  tables  in  enclosure  3  identify  the  statutory  and  regu¬ 
latory  requirements  that  shall  be  met  at  Milestone  B.  For  Shipbuilding  Pro¬ 
grams,  the  required  program  information  shall  be  updated  in  support  of  the 
Milestone  B  decision,  and  the  ICE  shall  be  completed.  The  lead  ship  in  a  class 
shall  normally  be  authorized  at  Milestone  B.  Technology  readiness  assess¬ 
ments  shall  consider  the  risk  associated  with  critical  subsystems  prior  to  ship 
installation.  Long  lead  for  follow  ships  may  be  initially  authorized  at  Milestone 
B,  with  final  authorization  and  follow  ship  approval  by  the  MDA  dependent  on 
completion  of  critical  subsystem  demonstration  and  an  updated  assessment  of 
technology  maturity. 

3.7.2.  Entrance  Criteria.  Entrance  into  this  phase  depends  on  technology 
maturity  (including  software),  approved  requirements,  and  funding.  Unless 
some  other  factor  is  overriding  in  its  impact,  the  maturity  of  the  technology 
shall  determine  the  path  to  be  followed.  Programs  that  enter  the  acquisition 
process  at  Milestone  B  shall  have  an  ICD  that  provides  the  context  in  which  the 
capability  was  determined  and  approved,  and  a  CDD  that  describes  specific 
program  requirements. 

3.7.2.2.  The  management  and  mitigation  of  technology  risk,  which  allows  less 
costly  and  less  time-consuming  systems  development,  is  a  crucial  part  of  over¬ 
all  program  management  and  is  especially  relevant  to  meeting  cost  and  schedule 
goals.  Objective  assessment  of  technology  maturity  and  risk  shall  be  a  routine 
aspect  of  DoD  acquisition.  Technology  developed  in  S&T  or  procured  from 
industry  or  other  sources  shall  have  been  demonstrated  in  a  relevant  environ¬ 
ment  or,  preferably,  in  an  operational  environment  to  be  considered  mature 
enough  to  use  for  product  development  in  systems  integration.  Technology 
readiness  assessments,  and  where  necessary,  independent  assessments,  shall  be 
conducted.  If  technology  is  not  mature,  the  DoD  Component  shall  use  alterna¬ 
tive  technology  that  is  mature  and  that  can  meet  the  user’s  needs. 

•  Enclosure  3:  Statutory,  Regulatory,  and  Contract  Reporting  Informa¬ 
tion  and  Milestone  Requirements 

E.3.1.  Tables  E3.T1,  E3.T2,  and  E3.T31,  below,  show  the  information  require¬ 
ments  for  all  milestones  and  phases,  both  statutory  and  regulatory,  to  include 
contract  reporting.  MDAs  may  tailor  regulatory  program  information  to  fit  the 
particular  conditions  of  an  individual  program.  A  non-mandatory  guidebook 


The  parts  of  Tables  E3.T1  and  E3.T2  relevant  to  this  discussion  are  included.  Table  E3.T3  is  not 
included  in  this  appendix  (Appendix  B  to  the  TRA  Deskbook). 


Table  E3.T1.  Statutory  Information  Requirements 


Information  Required 

Applicable  Statute 

When  Required 

The  following  information  requirements  are  statutory  for  both  MDAPs  and  MAIS  acquisition 
programs 

Consideration  of  Technology  Issues 

10  U.S.C.  2364,  reference  (q) 

Milestone  (MS)  A 

MSB 

MS  C 

The  following  information  requirements  are  statutory  for  MDAPs 
acquisition  programs  by  this  Instruction 

and  are  applicable  to  MAIS 

Technology  Development  Strategy 

Sec.  803,  Pub.L.  107-314, 

MSA 

(TDS) 

reference  (an) 

MSB 

MS  C 

Table  E3.T2.  Regulatory  Information  Requirements 


Information  Required 

Source 

When  Required 

Technology  Readiness  Assessment 

This  Instruction 

Program  Initiation  for  Ships 
(preliminary  assessment) 

MSB 

MS  C 

Independent  Technology 

This  Instruction 

MSB 

Assessment  (ACAT  ID  only) 

(if  required  by  DUSD(S&T)) 

MS  C 

Command,  Control,  Communications, 

DoD  Instruction  4630.8  and 

Program  Initiation  for  Ships 

Computers,  and  Intelligence  Support 

DoD  Directive  4630.5, 

MSB 

Plan  (C4ISP)  (also  summarized  in  the 
acquisition  strategy) 

references  (ar)  and  (as) 

MS  C 

shall  support  this  Instruction  to  provide  best  practices,  lessons  learned,  and 
expectations  for  the  information  required  by  these  tables.  Issues  regarding  the 
intent  of  the  expectations  described  in  the  guidebook  shall  be  resolved  by  the 
MDA.  The  AT&L  Knowledge  Sharing  System  (formerly  Defense  Acquisition 
Deskbook)  contains  a  library  of  mandatory  policy  and  regulations  and 
discretionary  practices  and  advice.  The  Internet  Web  site  address  is 
http://deskbook.dau.mil/. 

E.3.2.  The  following  Statutory  Information  Requirements  Table  is  divided  into 
sections  to  indicate  which  information  requirements  are  applicable  to  MDAPs, 
MAIS  programs,  or  both.  MAIS  programs  that  are  also  MDAPs  are  subject  to 
both  sets  of  statutory  requirements. 


B.1.3  INTERIM  DEFENSE  ACQUISITION  GUIDEBOOK,  DATED 
OCTOBER  30,  2002 

The  extracts  below  are  taken  directly  from  the  Interim  Guidebook  and  reflect  their 
origin  in  the  former  DoD  5000. 2-R.  While  using  these  extracts,  the  reader  should  keep  the 
FOREWORD  to  the  Guidebook  in  mind. 
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The  Deputy  Secretary’s  memorandum,  Defense  Acquisition,  dated  October 
30,  2002,  and  Attachment  2  to  that  memorandum  reference  a  guidebook  to 
accompany  the  interim  guidance.  The  former  DoD  5000. 2-R  regulation  will 
serve  as  the  guidebook  while  the  Defense  Acquisition  Policy  Working 
Group  creates  a  streamlined  guidebook.  The  former  DoD  5000. 2-R  is  NOT 
mandatory,  but  should  be  used  for  best  practices,  lessons  learned,  and 
expectations,  until  replaced. 

Subsequent  to  publication  of  the  streamlined  guidebook,  the  Technology  Readiness 
Assessment  (TRA)  Deskbook  will  be  updated  to  reflect  its  content. 

•  Technology  Maturity 

C7.5.1.  Technology  maturity  shall  measure  the  degree  to  which  proposed  criti¬ 
cal  technologies  meet  program  objectives.  Technology  maturity  is  a  principal 
element  of  program  risk.  A  technology  readiness  assessment  shall  examine 
program  concepts,  technology  requirements,  and  demonstrated  technology 
capabilities  to  determine  technological  maturity. 

C7.5.2.  The  PM  shall  identify  critical  technologies  via  the  WBS.  (See  para¬ 
graph  C5.3.1.)2  Technology  readiness  assessments  for  critical  technologies 
shall  occur  sufficiently  prior  to  milestone  decision  points  B  and  C  to  provide 
useful  technology  maturity  information  to  the  acquisition  review  process. 

C7.5.3.  The  DoD  Component  Science  and  Technology  (S&T)  Executive  shall 
direct  the  technology  readiness  assessment  and,  for  ACAT  ID  and  ACAT  IAM 
programs,  submit  the  findings  to  the  CAE  who  shall  submit  his  or  her  report  to 
the  DUSD(S&T)  with  a  recommended  technology  readiness  level  (TRL)  (or 
some  equivalent  assessment)  for  each  critical  technology.  When  the  Compo¬ 
nent  S&T  Executive  submits  his  or  her  findings  to  the  CAE,  he  or  she  shall 
provide  the  DUSD(S&T)  an  information  copy  of  those  findings.  In  coopera¬ 
tion  with  the  Component  S&T  Executive  and  the  program  office,  the 
DUSD(S&T)  shall  evaluate  the  technology  readiness  assessment  and,  if  he/she 
concurs,  forward  findings  to  the  OIPT  leader  and  DAB.  If  the  DUSD(S&T) 
does  not  concur  with  the  technology  readiness  assessment  findings,  an  inde¬ 
pendent  technology  readiness  assessment,  under  the  direction  of  the 
DUSD(S&T),  shall  be  required. 

C7.5.4.  TRL  descriptions  appear  at  Appendix  6. 3  TRLs  enable  consistent,  uni¬ 
form,  discussions  of  technical  maturity,  across  different  types  of  technologies. 
Decision  authorities  shall  consider  the  recommended  TRLs  (or  some  equivalent 
assessment  methodology,  e.g.,  Willoughby  templates)  when  assessing  program 
risk.  TRLs  are  a  measure  of  technical  maturity.  They  do  not  discuss  the 


-  For  paragraph  C5.3.1,  see  Section  B.4  of  this  appendix. 

3  These  definitions  also  appeal'  in  Section  III  of  this  TRA  Deskbook. 
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probability  of  occurrence  (i.e.,  the  likelihood  of  attaining  required  maturity)  or 
the  impact  of  not  achieving  technology  maturity. 

•  Integrated  Product  Teams  (IPTs)  in  the  Oversight  and  Review  Process 

C7.6.4.4.  For  ACAT  ID  decision  points,  the  OIPT  leader  shall  provide  the 
DAB  chair,  principals,  and  advisors  an  integrated  assessment  using  information 
gathered  through  the  IPT  process.  The  leader’s  assessment  shall  focus  on  core 
acquisition  management  issues  and  shall  consider  independent  assessments, 
including  technology  readiness  assessments,  which  the  OIPT  members  nor¬ 
mally  prepare.  These  assessments  typically  occur  in  context  of  the  OIPT 
review,  and  shall  be  reflected  in  the  OIPT  leader’s  report.  There  shall  be  no 
surprises  at  this  point-all  team  members  shall  work  issues  in  real  time  and 
shall  be  knowledgeable  of  their  OIPT  leader’s  assessment.  OIPT  and  other 
staff  members  shall  not  require  the  PM  to  provide  pre-briefs  independent  of  the 
OIPT  process. 

C7.6.7.  Independent  Assessments.  Assessments,  independent  of  the  developer 
and  the  user,  ensure  an  impartial  evaluation  of  program  status.  Consistent  with 
statutory  requirements  and  good  management  practice,  the  Department  of 
Defense  shall  require  independent  assessments  of  program  status  (e.g.,  the 
independent  cost  estimate  or  technology  readiness  assessment).  Senior  acqui¬ 
sition  officials  shall  consider  these  assessments  when  making  acquisition  deci¬ 
sions.  Staff  offices  that  provide  independent  assessments  shall  support  the 
orderly  and  timely  progression  of  programs  through  the  acquisition  process. 
IPTs  shall  have  access  to  independent  assessments  to  enable  full  and  open  dis¬ 
cussion  of  issues. 

B.2  EXTRACTS  FROM  THE  INTERIM  DEFENSE  ACQUISITION 
GUIDEBOOK  THAT  SUGGEST  TRA  RESPONSIBILITIES 

B.2.1  Program  Manager  (PM) 

C7.3.1.4.  The  PM  shall  brief  the  acquisition  program  to  the  DAB  and  specifi¬ 
cally  emphasize  technology  maturity,  risk  management,  affordability,  critical 
program  information,  technology  protection,  and  rapid  delivery  to  the  user.  The 
PM  shall  address  any  interoperability  and  supportability  requirements  linked  to 
other  systems,  and  indicate  whether  those  requirements  will  be  satisfied  by  the 
acquisition  strategy  under  review.  If  the  program  is  part  of  a  system-of-systems 
architecture,  the  PM  shall  brief  the  DAB  in  that  context.  If  the  architecture 
includes  less  than  ACAT  I  programs  that  are  key  to  achieving  the  expected 
operational  capability,  the  PM  shall  also  discuss  the  status  of  and  dependence 
on  those  programs. 
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C7.3.2.3.  Principal  participants  at  DoD  CIO  reviews  shall  include  (as  appropri¬ 
ate  to  the  issue  being  examined)  the  following  department  officials:  the  Deputy 
DoD  CIO;  IT  OIPT  Leader;  ACAT  ID  OIPT  Leaders;  Cognizant  PEO(s)  and 
PM(s);  Cognizant  OSD  PSA;  CAEs  and  CIOs  of  the  Army,  the  Navy,  and  the 
Air  Force.  Participants  shall  also  include  (as  appropriate  to  the  issue  being 
examined)  executive-level  representatives  from  the  following  organizations: 
Office  of  USD(AT&L);  Office  of  the  Under  Secretary  of  Defense  (Comptrol¬ 
ler);  Office  of  the  Joint  Chiefs  of  Staff;  Office  of  DOT&E;  Office  of  the 
Director,  PA&E;  and  Defense  Information  Systems  Agency. 

C7.5.2.  The  PM  shall  identify  critical  technologies  via  the  WBS.  (See  para¬ 
graph  C5.3.1.)4  Technology  readiness  assessments  for  critical  technologies 
shall  occur  sufficiently  prior  to  milestone  decision  points  B  and  C  to  provide 
useful  technology  maturity  information  to  the  acquisition  review  process. 

C7.6.4.1.  All  ACAT  ID  and  IAM  programs  shall  have  an  OIPT  to  provide 
assistance,  oversight,  and  review  as  the  program  proceeds  through  its  acquisi¬ 
tion  life  cycle.  An  appropriate  official  within  OSD,  typically  the  Director  of 
Strategic  and  Tactical  Systems  or  the  Principal  Director,  Command,  Control, 
Communications,  Intelligence,  Surveillance,  and  Reconnaissance  &  Space,  shall 
lead  the  OIPT  for  ACAT  ID  programs.  The  Deputy  DoD  CIO  or  designee 
shall  lead  the  OIPT  for  ACAT  IAM  programs.  The  OIPT  for  ACAT  IAM  pro¬ 
grams  is  called  the  IT  OIPT.  OIPTs  shall  comprise  the  PM,  PEO,  DoD  Com¬ 
ponent  Staff,  Joint  Staff,  and  OSD  staff  involved  in  oversight  and  review  of  the 
particular  ACAT  ID  or  IAM  program. 

C7.6.5.1.  The  PM,  or  designee,  shall  form  and  lead  an  IIPT  to  support  the 
development  of  strategies  for  acquisition  and  contracts,  cost  estimates,  evalua¬ 
tion  of  alternatives,  logistics  management,  training,  cost-performance  trade-offs, 
etc.  The  PM,  assisted  by  the  IIPT,  shall  develop  and  propose  to  the  OIPT,  a 
WIPT  structure.  The  IIPT  shall  coordinate  the  activities  of  the  WIPTs  and 
review  issues  they  do  not  address.  WIPTs  shall  meet  as  required  to  help  the 
PM  plan  program  structure  and  documentation  and  resolve  issues.  While  there 
is  no  one-size-fits-all  WIPT  approach,  the  following  basic  tenets  shall  apply: 

C7.6.5.1.1.  The  PM  is  in  charge  of  the  program. 

C7.6.5.1.2.  IPTs  are  advisory  bodies  to  the  PM. 

C76.5.1.3.  Direct  communication  between  the  program  office  and  all  levels  in 
the  acquisition  oversight  and  review  process  is  expected  as  a  means  of 
exchanging  information  and  building  trust. 


4  For  paragraph  C5.3.1,  see  Section  B.4  of  this  appendix. 
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C7.7.1.  It  shall  be  DoD  policy  to  keep  reporting  requirements  to  a  minimum. 
Nevertheless,  complete  and  current  program  information  is  essential  to  the 
acquisition  process.  Consistent  with  the  tables  of  required  regulatory  and 
statutory  information  appearing  in  reference  (a),  decision  authorities  shall 
require  PMs  and  other  participants  in  the  defense  acquisition  process  to  present 
only  the  minimum  information  necessary  to  understand  program  status  and 
make  informed  decisions.  The  MDA  shall  “tailor-in”  program  information 
case-by-case,  as  necessary.  IPTs  shall  facilitate  the  management  and  exchange 
of  program  information. 

C7.14.1.  PMs  shall  implement  internal  management  controls  in  accordance 
with  DoD  Directive  5000.1  (reference  (di)),  DoD  Instruction  5000.2  (refer¬ 
ence  (a)),  this  Regulation,  and  DoD  Directive  5010.38  (reference  (dj)).  APB 
parameters  shall  serve  as  control  objectives.  PMs  shall  identify  deviations  from 
approved  APB  parameters  and  exit  criteria  as  materiel  weaknesses.  PMs  shall 
focus  on  results,  not  process. 

C7.15.1.1.  Program  plans  describe  the  detailed  activities  of  the  acquisition  pro¬ 
gram.  In  coordination  with  the  PEO,  the  PM  shall  determine  the  type  and  num¬ 
ber  of  program  plans  needed  to  manage  program  execution. 

B.2.2  Deputy  Under  Secretary  of  Defense  for  Science  and  Technology 
DUSD(S&T) 

C7.5.3.  The  DoD  Component  Science  and  Technology  (S&T)  Executive  shall 
direct  the  technology  readiness  assessment  and,  for  ACAT  ID  and  ACAT  IAM 
programs,  submit  the  findings  to  the  CAE  who  shall  submit  his  or  her  report  to 
the  DUSD(S&T)  with  a  recommended  technology  readiness  level  (TRL)  (or 
some  equivalent  assessment)  for  each  critical  technology.  When  the  Compo¬ 
nent  S&T  Executive  submits  his  or  her  findings  to  the  CAE,  he  or  she  shall 
provide  the  DUSD(S&T)  an  information  copy  of  those  findings.  In  coopera¬ 
tion  with  the  Component  S&T  Executive  and  the  program  office,  the 
DUSD(S&T)  shall  evaluate  the  technology  readiness  assessment  and,  if  he/she 
concurs,  forward  findings  to  the  OIPT  leader  and  DAB.  If  the  DUSD(S&T) 
does  not  concur  with  the  technology  readiness  assessment  findings,  an  inde¬ 
pendent  technology  readiness  assessment,  under  the  direction  of  the 
DUSD(S&T),  shall  be  required. 

C7.6.7.  Independent  Assessments.  Assessments,  independent  of  the  developer 
and  the  user,  ensure  an  impartial  evaluation  of  program  status.  Consistent  with 
statutory  requirements  and  good  management  practice,  the  Department  of 
Defense  shall  require  independent  assessments  of  program  status  (e.g.,  the 
independent  cost  estimate  or  technology  readiness  assessment).  Senior  acqui¬ 
sition  officials  shall  consider  these  assessments  when  making  acquisition 
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decisions.  Staff  offices  that  provide  independent  assessments  shall  support  the 
orderly  and  timely  progression  of  programs  through  the  acquisition  process. 
IPTs  shall  have  access  to  independent  assessments  to  enable  full  and  open  dis¬ 
cussion  of  issues. 

B.2.3  Component  Acquisition  Executive  (CAE) 

C7.5.3.  The  DoD  Component  Science  and  Technology  (S&T)  Executive  shall 
direct  the  technology  readiness  assessment  and,  for  ACAT  ID  and  ACAT  IAM 
programs,  submit  the  findings  to  the  CAE  who  shall  submit  his  or  her  report  to 
the  DUSD(S&T)  with  a  recommended  technology  readiness  level  (TRL)  (or 
some  equivalent  assessment)  for  each  critical  technology.  When  the  Compo¬ 
nent  S&T  Executive  submits  his  or  her  findings  to  the  CAE,  he  or  she  shall 
provide  the  DUSD(S&T)  an  information  copy  of  those  findings.  In  coopera¬ 
tion  with  the  Component  S&T  Executive  and  the  program  office,  the 
DUSD(S&T)  shall  evaluate  the  technology  readiness  assessment  and,  if  he/she 
concurs,  forward  findings  to  the  OIPT  leader  and  DAB.  If  the  DUSD(S&T) 
does  not  concur  with  the  technology  readiness  assessment  findings,  an  inde¬ 
pendent  technology  readiness  assessment,  under  the  direction  of  the 
DUSD(S&T),  shall  be  required. 

B.2.4  Component  Science  and  Technology  (S&T)  Executive 

C7.5.3.  The  DoD  Component  Science  and  Technology  (S&T)  Executive  shall 
direct  the  technology  readiness  assessment  and,  for  ACAT  ID  and  ACAT  IAM 
programs,  submit  the  findings  to  the  CAE  who  shall  submit  his  or  her  report  to 
the  DUSD(S&T)  with  a  recommended  technology  readiness  level  (TRL)  (or 
some  equivalent  assessment)  for  each  critical  technology.  When  the  Compo¬ 
nent  S&T  Executive  submits  his  or  her  findings  to  the  CAE,  he  or  she  shall 
provide  the  DUSD(S&T)  an  information  copy  of  those  findings.  In  coopera¬ 
tion  with  the  Component  S&T  Executive  and  the  program  office,  the 
DUSD(S&T)  shall  evaluate  the  technology  readiness  assessment  and,  if  he/she 
concurs,  forward  findings  to  the  OIPT  leader  and  DAB.  If  the  DUSD(S&T) 
does  not  concur  with  the  technology  readiness  assessment  findings,  an  inde¬ 
pendent  technology  readiness  assessment,  under  the  direction  of  the 
DUSD(S&T),  shall  be  required. 

B.2.5  Defense  Acquisition  Board  [Chaired  by  the  Under  Secretary  of  Defense  for 
Acquisition  Technology  and  Logistics  (USD(AT&L))] 

C7.3.1.1.  The  DAB  shall  advise  the  USD(AT&L)  on  critical  acquisition  deci¬ 
sions.  The  USD(AT&L)  shall  chair  the  DAB,  and  the  Vice  Chairman  of  the 
Joint  Chiefs  of  Staff  shall  serve  as  vice-chair.  DAB  membership  shall  comprise 


B-12 


the  following  executives:  Under  Secretary  of  Defense  (Comptroller);  Under 
Secretary  of  Defense  (Policy);  Under  Secretary  of  Defense  (Personnel  & 
Readiness);  ASD(C3I)/DoD  CIO;  DOT&E;  and  the  Secretaries  of  the  Army, 
the  Navy,  and  the  Air  Force.  United  States  Joint  Forces  Command  shall  be 
available  to  comment  on  interoperability  and  integration  issues  that  the  JROC 
forwards  to  the  DAB.  The  DAE  may  ask  other  department  officials  to  partici¬ 
pate  in  reviews,  as  required. 

B.2.6  Defense  Acquisition  Executive  (DAE) 

C7.3.1.3.  The  DAE  shall  conduct  DAB  reviews  at  major  program  milestones 
and  at  the  Full-Rate  Production  Decision  Review  (if  not  delegated  to  the  CAE), 
and  at  other  times,  as  necessary.  An  ADM  shall  document  the  decision(s) 
resulting  from  the  review. 

B.2.7  DoD  Chief  Information  Officer  (CIO)  Reviews 

C7.3.2.1.  DoD  CIO  Reviews  shall  provide  the  forum  for  ACAT  IAM  mile¬ 
stones,  for  deciding  critical  ACAT  IAM  issues  when  they  cannot  be  resolved  at 
the  OIPT  level,  and  for  enabling  the  execution  of  the  DoD  CIO’s  acquisition- 
related  responsibilities  for  IT,  including  NSS,  under  the  Clinger-Cohen  Act  and 
Title  10  U.S.C.  (references  (bn)  and  (dd)).  Wherever  possible,  these  reviews 
shall  take  place  in  the  context  of  the  existing  IPT  and  acquisition  milestone 
review  process.  Where  appropriate,  an  ADM  shall  typically  document  the  deci¬ 
sion^)  resulting  from  the  review. 

B.2.8  Overarching  Integrated  Product  Team  (OIPT) 

C7.6.4.1.  All  ACAT  ID  and  IAM  programs  shall  have  an  OIPT  to  provide 
assistance,  oversight,  and  review  as  the  program  proceeds  through  its  acquisi¬ 
tion  life  cycle.  An  appropriate  official  within  OSD,  typically  the  Director  of 
Strategic  and  Tactical  Systems  or  the  Principal  Director,  Command,  Control, 
Communications,  Intelligence,  Surveillance,  and  Reconnaissance  &  Space,  shall 
lead  the  OIPT  for  ACAT  ID  programs.  The  Deputy  DoD  CIO  or  designee 
shall  lead  the  OIPT  for  ACAT  IAM  programs.  The  OIPT  for  ACAT  IAM  pro¬ 
grams  is  called  the  IT  OIPT.  OIPTs  shall  comprise  the  PM,  PEO,  DoD  Com¬ 
ponent  Staff,  Joint  Staff,  and  OSD  staff  involved  in  oversight  and  review  of  the 
particular  ACAT  ID  or  IAM  program. 

C7.6.4.2.  The  OIPT  shall  form  upon  departmental  intention  to  start  an  acquisi¬ 
tion  program.  The  OIPT  shall  charter  the  IIPT  and  WIPTs.  The  OIPT  shall 
consider  the  recommendations  of  the  IIPT  regarding  the  appropriate  milestone 
for  program  initiation  and  the  minimum  information  needed  for  the  program 
initiation  milestone  review.  OIPTs  shall  meet,  thereafter,  as  necessary  over  the 
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life  of  the  program.  The  OIPT  leader  shall  act  to  resolve  issues  when  requested 
by  any  member  of  the  OIPT,  or  when  so  directed  by  the  MDA.  The  goal  is  to 
resolve  as  many  issues  and  concerns  at  the  lowest  level  possible,  and  to  expedi¬ 
tiously  escalate  issues  that  need  resolution  at  a  higher  level.  The  OIPT  shall 
bring  only  the  highest-level  issues  to  the  MDA  for  decision. 

C7.6.4.3.  The  OIPT  shall  normally  convene  2  weeks  before  a  planned  decision 
point.  It  shall  assess  the  information  and  recommendations  that  the  MDA  will 
receive,  in  the  same  context,  and  to  the  same  ACAT  level.  It  shall  also  assess 
family-of-system  or  system-of- system  capabilities  within  mission  areas  in  sup¬ 
port  of  mission  area  operational  architectures  developed  by  the  Joint  Staff.  If 
the  program  includes  a  pilot  project,  such  as  TOC  Reduction,  the  PM  shall 
report  the  status  of  the  project  to  the  OIPT.  The  OIPT  shall  then  assess  pro¬ 
gress  against  stated  goals.  The  PM’s  briefing  to  the  OIPT  shall  specifically 
address  interoperability  and  supportability  (including  spectrum  supportability) 
with  other  systems,  anti-tamper  provisions,  and  indicate  whether  those  require¬ 
ments  will  be  satisfied  by  the  acquisition  strategy  under  review.  If  the  program 
is  part  of  a  family-of-systems  architecture,  the  PM  shall  brief  the  OIPT  in  that 
context.  If  the  architecture  includes  less  than  ACAT  I  programs  that  are  key  to 
achieving  the  expected  operational  capability,  the  PM  shall  also  discuss  the 
status  of  and  dependence  on  those  programs.  The  OIPT  leader  shall  recom¬ 
mend  to  the  MDA  whether  the  anticipated  review  should  go  forward  as 
planned. 

C7.6.4.4.  For  ACAT  ID  decision  points,  the  OIPT  leader  shall  provide  the 
DAB  chair,  principals,  and  advisors  an  integrated  assessment  using  information 
gathered  through  the  IPT  process.  The  leader’s  assessment  shall  focus  on  core 
acquisition  management  issues  and  shall  consider  independent  assessments, 
including  technology  readiness  assessments,  which  the  OIPT  members  nor¬ 
mally  prepare.  These  assessments  typically  occur  in  context  of  the  OIPT 
review,  and  shall  be  reflected  in  the  OIPT  leader’s  report.  There  shall  be  no 
surprises  at  this  point-all  team  members  shall  work  issues  in  real  time  and 
shall  be  knowledgeable  of  their  OIPT  leader’s  assessment.  OIPT  and  other 
staff  members  shall  not  require  the  PM  to  provide  pre-briefs  independent  of  the 
OIPT  process. 

B.2.9  Integrated  Product  Teams  (IPTs) 

C7.6.2.  IPTs  are  an  integral  part  of  the  defense  acquisition  oversight  and  review 
process.  For  ACAT  ID  and  IAM  programs,  there  are  generally  two  levels  of 
IPT:  the  OIPT  and  WIPT(s).  Each  program  shall  have  an  OIPT  and  at  least 
one  WIPT.  WIPTs  shall  focus  on  a  particular  topic  such  as  cost/performance, 
test,  or  contracting.  An  Integrating  IPT  (IIPT)  (which  is  a  WIPT)  shall  coordi¬ 
nate  WIPT  efforts  and  cover  all  topics  not  otherwise  assigned  to  another  IPT. 
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IPT  participation  is  the  primary  way  for  any  organization  to  participate  in  the 
acquisition  program. 

B.2.10  Authority  of  Key  Acquisition  System  Officials:  From  DoDD  5000.1 

5.1.  The  USD(AT&L),  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications,  and  Intelligence),  and  the  Director  of  Operational  Test  and 
Evaluation  are  key  officials  of  the  Defense  Acquisition  System.  Consistent  with 
their  respective  authorities,  they  may  jointly  issue  DoD  Instructions,  DoD  Pub¬ 
lications,  and  one-time  directive-type  memoranda,  consistent  with  DoD  5025.1- 
M  (reference  (c)),  that  implement  the  policies  contained  in  this  Directive. 
Financial  Management  Enterprise  Architecture  (FMEA)  Requirements  shall  be 
addressed  for  all  financial  management  and  mixed  (financial  and  non-financial) 
information  systems  and  shall  be  certified  as  being  compliant  with  the  FMEA 
by  the  Under  Secretary  of  Defense  (Comptroller)  (USD(C)). 

B.3  A  COMMENT  ON  THE  TRA  PROCESS 

The  Interim  Defense  Acquisition  Guidebook  (October  30,  2002) 

Appendix  6  of  the  Interim  Defense  Acquisition  Guidebook  lists  9  TRFs  and  their 
definitions.5  The  specific  level  for  passing  MS  B  and  MS  C  is  not  directed.  Nonetheless, 
the  wording  “...  in  an  appropriate  simulated  environment,  or  preferably  in  an  operational 
environment”  strongly  suggests  TRF  6  or  TRF  7  at  MS  B . 

B.4  REFERENCES 

•  From  DoDI  5000.2,  Operation  of  the  Defense  Acquisition  System,  dated 
May  12,  2003  (see  Paragraph  B.1.2  of  this  appendix) 

-  (g)  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170.01  Series, 
“Requirements  Generation  System,”  April  15,  2001. 

-  (q)  Section  2364  of  title  10,  United  States  Code,  “Coordination  and 
Communication  of  Defense  Research  Activities.” 

-  (an)  Section  803,  Public  Faw  107-314,  “Bob  Stump  National  Defense 
Authorization  Act  for  Fiscal  Year  2003,”  “Spiral  development  under 
major  defense  acquisition  programs.” 


5  These  definitions  also  appear  in  Section  III  of  this  TRA  Deskbook. 
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-  (ar)  DoD  Instruction  4630.8,  “Procedures  for  Interoperability  and  Sup- 
portability  of  Information  Technology  (IT)  and  National  Security  Systems 
(NSS),”  May  2,  2002. 

-  (as)  DoD  Directive  4630.5,  “Interoperability  and  Supportability  of  Infor¬ 
mation  Technology  (IT)  and  National  Security  Systems  (NSS),”  January 
11,2002. 

•  From  the  Interim  Defense  Acquisition  Guidebook,  dated  October  30,  2002 
(see  Paragraph  B.2.1  of  this  appendix) 

-  (a)  DoD  Instruction  5000.2,  “Operation  of  the  Defense  Acquisition  Sys¬ 
tem,”  April  5,  2002. 

-  (di)  DoD  Directive  5000.1,  “The  Defense  Acquisition  System,”  October 
23,  2000. 

-  (dj)  DoD  Directive  5010.38,  “Management  Control  (MC)  Program,” 
August  26,  1996. 

•  From  the  Interim  Defense  Acquisition  Guidebook,  dated  October  30,  2002 
(see  paragraphs  B.1.3  and  B.2.1  of  this  appendix) 

-  Paragraph  C5.3.1.  Work  Breakdown  Structure  (WBS).  Systems  engi¬ 
neering  shall  yield  a  program  WBS.  The  PM  shall  prepare  the  WBS  in 
accordance  with  the  WBS  guidance  in  MIL-HDBK-881  (reference  (cj)). 
The  WBS  provides  the  framework  for  program  and  technical  planning, 
cost  estimating,  resource  allocation,  performance  measurement,  technical 
assessment,  and  status  reporting.  The  WBS  shall  include  the  WBS  dic¬ 
tionary.  The  WBS  shall  define  the  system  to  be  developed  or  produced.  It 
shall  display  the  system  as  a  product-oriented  family  tree  composed  of 
hardware,  software,  services,  data,  and  facilities.  It  shall  relate  the  elements 
of  work  to  each  other  and  to  the  end  product.  The  PM  shall  normally 
specify  contract  WBS  elements  only  to  level  three  for  prime  contractors 
and  key  subcontractors.  Only  low-level  elements  that  address  high-risk, 
high-value,  or  high-technical-interest  areas  of  a  program  shall  require 
detailed  reporting  below  level  three.  The  PM  shall  have  only  one  WBS  for 
each  program. 
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ACRONYMS  AND  ABBREVIATIONS  FOR  APPENDIX  B 


ACAT 

Acquisition  Category 

ADM 

Acquisition  Decision  Memorandum 

AoA 

Analysis  of  Alternatives 

APB 

Acquisition  Program  Baseline 

ASD(C3I) 

Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications,  and  Intelligence 

AT&L 

Acquisition,  Technology,  and  Logistics 

C4ISP 

Command,  Control,  Communications,  Computers,  and 
Intelligence  Support  Plan 

CAE 

Component  Acquisition  Executive 

CDD 

Capability  Development  Document 

CIO 

Chief  Information  Officer 

COTS 

commercial-off-the-shelf 

DAB 

Defense  Acquisition  Board 

DAE 

Defense  Acquisition  Executive 

DDL 

Delegation  of  Disclosure  Authority  Letter 

DoD 

Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

DOT&E 

Director  of  Operational  Test  and  Evaluation 

DOTMLPF 

doctrine,  organization,  training,  materiel,  leadership, 
personnel,  and  facilities 

DUSD(S&T) 

Deputy  Under  Secretary  of  Defense  for  Science  and 
Technology 

FMEA 

Financial  Management  Enterprise  Architecture 

ICD 

Initial  Capabilities  Document 

ICE 

independent  cost  estimate 

IIPT 

Integrating  IPT 

IPT 

Integrated  Product  Team 

IT 

Information  Technology 

JROC 

Joint  Requirements  Oversight  Committee 

MAIS 

Major  Automated  Information  System 

MDA 

Milestone  Decision  Authority 

MDAP 

Major  Defense  Acquisition  Program 
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MS 

Milestone 

NSS 

National  Security  Systems 

OIPT 

Overarching  Integrated  Product  Team 

OSD 

Office  of  the  Secretary  of  Defense 

PA&E 

Program  Analysis  and  Evaluation 

PEO 

Program  Executive  Officer 

PM 

Program  Manager 

PSA 

Principal  Staff  Assistant 

S&T 

Science  and  Technology 

SDD 

System  Development  and  Demonstration 

TDS 

Technology  Development  Strategy 

TOC 

total  ownership  cost 

TRA 

Technology  Readiness  Assessment 

TRL 

Technology  Readiness  Level 

use 

United  States  Code 

USD(AT&L) 

Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics 

USD(C) 

Under  Secretary  of  Defense  (Comptroller) 

WBS 

Work  Breakdown  Structure 

WIPT 

Working  Integrated  Product  Team 
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APPENDIX  C 

POLICY  STATEMENT 


A  congressional  directive  is  provided  here. 


National  Defense  Authorization  Act  (NDAA)  for  Fiscal  Year  2002, 
SEC.  804.  Reports  on  Maturity  of  Technology  at  Initiation  of  Major 
Defense  Acquisition  Programs . . 
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C-l 


tOTin  Congress  (rPPn1Milwt*TWM  *“™* 

J;f  Sniion  HOUSE  OF  REPRESENTATIVES  107-333 


NATIONAL  DEFENSE  AUTHORIZATION 
ACT  FOR  FISCAL  YEAR  2Q02 


CONFERENCE  REPORT 

TO  ACCDHFAWY 

5.  1436 


DEOEH&ER  12r  2001. — Ordered  ta  be  prinSed 
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sec.  ,5'dW.  RETORTS  ON  MATURITY  OF  TtCUNOLDCY  AT  TNTTIATTON  OF 

MAJOR  DlFtVi’i  ALTjlMinOA'  PROGRAMS. 

(a:  Efforts  Required. — Not  laier  than  AfartA  I  eo,th  of 
ye&TS  £003  iknwgh  2006,  the  Secretary  of  Defense  shall  tubviii  to 
the  Comm  itttes  an  Armed  Servian  of  ike.  Senate  aftitf  the  House  of 
Repiestntotioei  □  report  on  the  imptemmtalinn  of  the  req.:i  kltlrtffti 
jft  partfgniph  47.3.2.2.2  of  Department  of  Defeat  Instruction 
5QQQ.2,  its  Eft  effect  on.  the  date  of  enactment  of  this  Ait,  that  (eftfc- 
nology  i'1/ist  hose  fceerc  demonstrated  in  a  relevant  MviroilTiMf'lJ  (or} 
preferably,  in  an  opcraitorzcl  environment)  to  he  considered  mature 
enough  Ilf  ttee  for  product  ^EL,-m'-s;p.'iLE.'i£  in  .^fEnm.-r  Lrkffjnifkja. 

(b)  Cunte.'/ts  of  Reports.— Ewh  report  required  by  jiitserfMWi 
fa)  shall — 

(I)  identify  efleft  case  JFl  tvh  kA  a  JttfljiflP  defense  acquisition 
program  entered  system  dtudopinunt  and  demonstration  daring 
the  preceding  calendar  year  and  irafo  iaAicA  try  ledhnoiog y  has 
been  incorporated  that  does  not  meet  the  technological  maturity 
requ  irttfteftt  dmrrbvd  in  subsection  (a),  and  provide  a  juitifKa- 

ttOft  for  StA;y  such  key  technology  mar  incorporated;  and 

(2j  identify  any  determination  of  rfL'AnaukkccJ  /naiarify 
MitfA  which  the  Deputy  Under  SecTTEhuy  of  Defense  for  Science 
and  Technology  did  not  concur  anrf  eatjuiain  A«n  the  irruE  Aar 
foffc.i  or  will  Ac  resolved. 

(e)  MAJOR  Defense  Acquisition  Program  Defined. — In  (A+s- 
recfiOli,  (ft?  term  'major  defense  ncijuirilum  program'  has  the  m-ran- 
mg  given  that  term  tr»  sEtEiun  139(a) fB)  of  title  d  States 

Code- 
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APPENDIX  D 

TECHNOLOGY  READINESS  LEVEL  (TRL)  EXAMPLES 


Table  III- 1  of  the  TRA  Deskbook  contains  the  definitions  of  the  various  TRLs  and 
notes  some  of  the  information  that  supports  assignment  of  a  technology  to  specific  levels  of 
readiness.  To  aid  in  making  the  definitions  more  concrete,  this  appendix  contains  several 
examples  of  readiness  levels  for  technologies  as  the  evolved  to  full  maturity. 


Ring  Laser  Gyro6  .  D-3 

Technology  Steel  Readiness  Levels  Example:  HSLA-100  Steel  for  Aircraft 

Carrier  Structure7 .  D-13 

Acronyms  and  Abbreviations  for  Appendix  D .  D-33 


6  Compliments  of  the  Army,  in  which  the  evolution  of  a  technology  is  depicted  graphically. 

7  Compliments  of  the  Navy,  in  which  the  evolution  of  a  materials  technology  is  presented,  with  a  full 
description  at  each  TRL. 
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RING  LASER  GYRO 
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Technology  Readiness  Example 
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HG1108  Inertial  Measurement  Unit  circa  1990 


Technology  Readiness  Example 
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Technology  Readiness  Example 
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Technology  Readiness  Example 
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Technology  Readiness  Example 
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TECHNOLOGY  STEEL  READINESS  LEVELS 
EXAMPLE:  HSLA-100  STEEL  FOR  AIRCRAFT  CARRIER  STRUCTURE 

MARCH  2002 
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Technology  Readiness  Level  1: 

Basic  Principles  Observed  and  Reported 

The  lowest  Technology  Readiness  Level  (TRL),  where  scientific  research 
begins  to  be  translated  into  technology’s  basic  properties. 

With  the  mass  industrialization  of  structural  steel  welding  for  shipbuilding  in  World 
War  II,  the  quest  for  high-strength  steels  with  good  weldability  was  a  motivation  for 
metallurgical  research  that  continued  through  the  post-war  era.  Carbon  strengthening  and 
alloying  that  resulted  in  high  strength  was  counter  to  weldability.  The  fundamental 
metallurgical  tools  for  steel  alloy  design  (e.g.,  phase  transformation,  phase  diagrams, 
relationship  of  microstructure  to  properties,  precipitation  strengthening,  and  so  forth)  were 
developing  at  a  dramatic  rate  along  with  the  U.S.  steel  industry. 

In  the  1930s,  the  unique  property  of  precipitation  hardening  induced  by  alloying  of 
copper  in  steel  was  established.  The  phase  diagrams  for  the  Fe-Cu  system  were  formulated, 
the  solubility  limits  of  Cu  in  low  carbon  steel  were  explored,  and  laboratory  studies  of 
copper  steels  were  conducted.  However,  the  benefit  of  Cu-strengthening  as  a  means  toward 
optimum  strength,  toughness,  and  weldability  was  not  recognized. 

Key  References: 

Smith,  C.S.  and  E.W.  Palmer,  “The  Precipitation-hardening  of  Copper  Steels,”  Trans. 
A1ME,  Vol.  105  (1933). 


Fe-Cu  Phase  Diagram 


Precipitation  Hardening  in  Heat 
Treatment  of  an  0.27%  C,  1%  Cu  Steel 
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Technology  Readiness  Level  2: 

Invention  Begins 

Once  basic  principles  are  observed,  practical  applications  can  be  invented. 

However,  the  application  is  speculative,  and  no  proof  or  detailed  analysis 

exists  to  support  the  assumption. 

In  the  mid-1960s,  the  laboratories  of  the  International  Nickel  Company  (INCO)  ini¬ 
tiated  the  development  of  a  class  of  low-carbon,  age-hardening  Ni-Cu-Cb  steels  called 
“NiCuAge”  steels.  The  work  focused  on  the  very  low  carbon,  with  changes  in  Ni,  Cu,  and 
Cb  content  and  processing  (hot  working  schedules  and  heat  treatment)  to  establish  micro¬ 
structure-mechanical  property  relationships.  The  combinations  of  strength,  ductility,  and 
processing  characteristics  exhibited  by  the  Ni-Cu-Cb  steels  suggested  a  variety  of  applica¬ 
tions  in  transportation,  automotive,  and  oil  field  construction.  Because  of  the  low  carbon 
content,  the  steel  offered  excellent  formability  and  weldability  in  the  fully  strengthened 
condition. 

The  key  concepts  discovered  at  this  stage  were  the  importance  of  Ni  and  Cb  addi¬ 
tions  to  the  copper  steels.  The  Ni  addition  and  the  ratio  of  Ni-to-Cu  were  established  as  a 
means  to  prevent  cracking  during  hot  working.  Researchers  discovered  that  small  additions 
of  Cb  significantly  increased  strength,  provided  grain  refinement,  and  did  not  degrade  any 
characteristics  of  the  steel.  At  this  stage,  small  laboratory  melts  (30  lb)  were  used  for  the 
alloy  composition  optimization. 

Key  References: 

Hurley,  J.L.  and  C.H.  Shelton,  “Age-Hardenable  Nickel-Copper  Steels,”  Metals  Engi¬ 
neering  Quarterly,  ASM,  May  1966. 


Tensile  Ductility  of  Ni-Cu 
Steel  as  Influenced  by 
Carbon  Content 
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Technology  Readiness  Level  3: 

Active  Research  and  Development  (R&D)  Is  Initiated 

This  includes  analytical  and  laboratory  studies  to  validate  physically  the 

analytical  predictions  of  separate  elements  of  the  technology. 

INCO  continued  the  development  of  improved  “NiCuAge”  steel  for  improved 
weldability  and  low- temperature  toughness  in  heavy  section  plates  and  forgings  and,  in 
1972,  marketed  the  steel  designated  IN-787  for  offshore  platforms  and  ship  hull  plates.  The 
American  Society  for  Testing  and  Materials  (ASTM)  Standard  Specification  A710, 
Grade  A,  based  on  IN-787  steel,  was  issued  in  1975.  Armco  Steel  Corporation  produced  a 
plate  to  ASTM  A710,  Grade  A,  under  the  trade  name  “NI-COP”  steel. 

The  primary  reason  for  preheat  in  the  welding  of  High  Yield  Strength  (HY)-80  and 
HY-100  steels  is  to  mitigate  underbead  cracking  (hydrogen  related)  in  the  hard,  martensitic 
heat-affected  zone  (HAZ).  The  Navy  High-Strength  Low-Alloy  (HSLA)-80,  an  optimized 
version  of  ASTM  A710,  Grade  A  steel,  is  a  ferritic  steel.  The  micro  structure  of  the 
quenched  and  aged  HSLA-80  plate  product  is  generally  an  acicular  ferrite.  Ferritic  steels  are 
widely  used  in  civil  construction  because  of  their  excellent  weldability. 

In  1981,  the  Navy  HSLA  Steels  Exploratory  Development  Program  was  initiated  at 
David  Taylor  Research  Center  (DTRC),  with  ASTM  A710,  Grade  A  selected  as  the  primary 
candidate.  Because  of  the  positive  results  emanating  from  the  project,  ASTM  A710,  Grade 
A,  Class  3  steel  was  authorized  as  substitute  for  HY-80  steel  on  a  production  trial  basis  in 
CVN  71  in  selected  noncritical,  nonwetted  areas  in  1983.  Upon  completion  of  the  evaluation 
of  ASTM  A710  for  Navy  requirements,  the  modifications  to  ASTM  A710  were 
incorporated  in  MIL-S-24645(SH),  4  September  1984,  for  HSLA-80  steel  plate,  sheet,  and 
coil.  The  Naval  Sea  Systems  Command  (NAVSEA)  certified  HSLA-80  for  surface  ship 
construction  and  repair  in  thickness  up  to  1-1/4  inch,  16  February  1984.  The  evaluation  of 
HSLA-80  properties,  welding,  and  structural  performance  demonstrated  that  the  very-low- 
carbon,  copper  precipitation-strengthened  steel  met  the  requirements  of  HY-80  steel  and  was 
readily  weldable  with  no  preheat  (32  °F  minimum)  using  the  same  welding  consumables 
and  processes  as  those  used  for  HY-80  steel  fabrication.  Since  1985,  HSLA-80  steel  has 
been  used  in  CG  47  Class  construction  in  increasing  tonnage,  in  CVN  72  and  follow-on 
ships,  and  in  DDG  51  Class,  LHD  1  Class,  LSD  41  Class,  and  FFG  7  Class  modifications. 

Following  the  HSLA-80  program,  a  research  and  development  (R&D)  project  com¬ 
menced  in  1985  to  establish  the  feasibility  of  HSLA- 100  steel  as  a  replacement  for  HY-100 
to  reduce  fabrication  costs.  A  contract  to  AMAX  Materials  Research  Center  in  1985  initi¬ 
ated  the  laboratory  alloy  development  for  HSLA- 100  steel.  The  objective  for  HSLA- 100 
was  to  meet  the  strength  and  toughness  of  HY-100  steel  but  to  be  weldable  without  the  pre¬ 
heat  requirements  of  HY-100,  using  the  same  welding  consumables  and  processes  as  those 
used  in  welding  HY-100.  The  project  for  the  development  of  HSLA-100  steel  in  the  labo¬ 
ratory  alloy  design  phase  used  the  principles  of  very  low  carbon,  copper-precipitation 
strengthened  steel  successful  for  HSLA-80. 
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Fracture -process  research  on  HSLA-80  steel  indicated  that  a  uniformly  small  grain 
size  and  wider  distribution  of  small  carbides  would  reduce  the  fracture  transition  tempera¬ 
ture.  In  fact,  HSLA-80  plates  of  1-inch  gage  and  less  were  typically  a  fine-grained,  acicular 
ferrite  microstructure  with  widely  dispersed  fine  carbides  and  showed  excellent  low-tem¬ 
perature  toughness.  The  aim  of  HSLA-100  alloy  design  was  to  produce  a  homogeneous, 
fine-grained,  low-carbon  martensite  microstructure  that  dispersed  the  secondary  transfor¬ 
mation  products.  The  alloy  development  effort  to  modify  HSLA-80  steel  microstructurally 
used  laboratory-scale  heats  (50  to  100  lb)  to  study  the  effects  of  Mn,  Ni,  Mo,  Cu,  Cr,  Cb, 
and  C  in  hot  rolled,  quenched,  and  aged  HSLA-100  plate.  Laboratory  plates  in  thicknesses 
of  1/4,  3/4,  1-1/4,  and  2  inches  of  HSLA-100  exceeded  the  minimum  strength  and  impact 
toughness  requirements. 

Micro  structural  analysis  was  conducted  to  develop  composition  ranges  for  heavy 
gage  plate,  meeting  the  strength  and  toughness  requirements,  where  polygonal  (“blocky”) 
ferrite  microstructures  were  not  present.  A  regression  analysis  was  conducted  on  the  results 
for  plates  from  45  experimental  melts  to  develop  composition  ranges  for  an  Interim  Specifi¬ 
cation  for  HSLA-100  Steel  Plate.  The  Interim  Specification  was  then  used  as  the  basis  for  a 
trial  commercial  production  of  HSLA-100  steel  by  domestic  steel  plate  mills. 

The  copper  content  of  HSLA-100  steel  is  higher  than  that  in  HSLA-80  [for  addi¬ 
tional  precipitation  strengthening  (maximum  solubility  of  copper  in  iron  is  near  2  percent)], 
and  increased  hardenability  was  achieved  by  increases  in  manganese,  nickel,  and  moly¬ 
bdenum.  Nickel,  the  greatest  increase  over  that  in  HSLA-80,  lowers  upper  shelf  impact 
toughness  but  also  lowers  (improves)  the  impact  toughness  transition  temperature.  The 
microstructure  of  HSLA-100  steel  was  identified  by  optical  and  scanning  electron  micros¬ 
copy  as  low-carbon  martensite  or  a  granular,  low-carbon  bainite,  depending  on  plate 
gage— a  significantly  different  metallurgy  and  microstructure  than  the  ferritic  HSLA-80 
steel  microstructures. 
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Thickness 


Technology  Readiness  Level  4: 

Basic  Technology  Components  Are  Integrated 

The  basic  components  of  the  technology  are  integrated  to  establish  that  the 

pieces  will  work  together. 

For  the  trial  plate  production  phase  of  the  HSLA-100  steel  project,  an  initial  150-ton 
production  of  HSLA-100  steel  was  melted  and  rolled  by  Phoenix  Steel  Corporation  in  1986 
to  the  interim  specification,  using  conventional  electric  furnace  and  ingot  casting  practice, 
conducted  to  achieve  a  very-low-carbon  composition.  The  minimum  strength  and  toughness 
requirements  of  the  interim  specification  were  met  in  the  initial  production  of  HSLA-100 
steel  plate  in  gages  from  1/4  to  2  inches.  Optimum  properties  in  HSLA-100  plate  resulted 
from  aging  temperatures  from  1150  to  1275  °F. 

Upon  receipt  of  HSLA-100  plate  from  the  trial  productions,  an  evaluation  com¬ 
menced  to  evaluate  HSLA-100  steel  plate  and  welding  using  the  processes  and  procedures 
for  HY-100  steel  ship  and  submarine  structural  applications— but  with  reduced  or  no  pre¬ 
heat.  The  evaluation  of  HSLA-100  steel  plate  properties  and  welding  demonstrated  that 
HSLA-100  steel  met  the  mechanical  property  requirements  of  HY-100  steel  and  was 
weldable  with  reduced  preheat  requirements,  using  the  same  welding  consumables  as  for 
HY-100  steel  fabrication.  When  compared  with  HY-100  steel,  the  tensile  and  impact  tough¬ 
ness  properties  of  the  plates  met  or  exceeded  the  requirements. 

The  primary  reason  for  preheating  when  welding  the  HY-series  steels  was  to  miti¬ 
gate  underbead  cracking  (hydrogen  related)  in  the  HAZ.  The  HSLA-100  precertification 
evaluation  emphasized  welding  and  weldability  testing  to  demonstrate  that  HSLA-100  was 
more  resistant  to  hydrogen  cracking  than  HY-100  (to  allow  a  relaxation  of  preheat  require¬ 
ments).  The  findings  of  the  HSLA-100  steel  welding  and  weldability  evaluations  are  sum¬ 
marized  as  follows: 

•  The  strength  and  toughness  of  weld  metals  deposited  by  the  Shielded  Metal 
Arc  Welding  (SMAW),  Submerged  Arc  Welding  (SAW),  Pulsed  Gas  Metal 
Arc  Welding  (GMAW-P),  and  Short  Circuiting  Gas  Metal  Arc  Welding 
(GMAW-S)  processes,  using  the  welding  consumables  qualified  for  HY-100 
welding,  met  the  requirements  when  welded  over  a  broader  range  of  operating 
conditions  (heat  inputs  ranging  from  22  to  65  kJ/in.)  than  for  HY-100.  No 
“hard”  microstructures  were  indicated,  and  the  Charpy  V-notch  toughness  of 
the  HAZ  in  HSLA-100  weldments  was  equal  to  or  greater  than  the  weld  metal 
toughness. 

•  It  was  demonstrated  that  HSLA-100  fillet  weld  strengths  were  equivalent  to 
HY-100  welds  using  the  same  process,  filler  metal,  and  fillet  size. 

•  HSLA-100  plate,  weld  metal,  and  weld  HAZ  did  not  show  any  susceptibility  to 
stress  corrosion  cracking  exposed  at  -1,000  mV  at  or  above  stress  corrosion 
cracking  threshold  stress  intensity  values  determined  for  HY-100,  MIL-100S-1, 
and  MIL-120S-1  weld  metals. 
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Fracture  Toughness  Test  Results  of  HSLA-100  and  HY-100 


Varestraint  Weldability  Tests  of  High-Strength  Steels 


Technology  Readiness  Level  5: 

Technology  Sufficiently  Advanced  For  Simulation  Tests 

The  fidelity  of  breadboard  technology  increases  significantly  enough  to  jus¬ 
tify  being  ready  for  testing  in  a  simulated  environment. 

Lukens  Steel  Company  produced  a  second  melt  of  HSLA-100  steel,  again  by  elec¬ 
tric  furnace  and  ingot  casting.  Most  of  the  plate  produced  from  the  heat  was  greater  than 
2  inches  thick,  primarily  for  ballistic  resistance  evaluation.  The  minimum  strength  and 
toughness  requirements  were  met  in  plate  thicknesses  from  1/2  to  3-3/4  inches.  A  double 
austenitization  and  quench  process  was  used  for  HSLA-100  steel  plate  in  gages  over 
1-1/4  inches  to  refine  the  heavy-plate  grain  structure  for  optimum  toughness.  HSLA-100 
plate  from  both  productions  to  the  interim  specification  was  the  primary  material  used  in  the 
certification  program. 

The  certification  evaluation  included  continued  characterization  of  production 
HSLA-100  steel  plate  mechanical,  physical,  and  fracture  properties.  However,  the  main 
focus  was  the  evaluation  of  weldability  and  welding  process  limits  for  structures  of  high 
restraint,  studies  of  fatigue  properties,  and  effects  of  marine  environments  on  HSLA-100. 

The  results  of  low-cycle  fatigue  crack  initiation  tests  of  HSLA-100  steel  and  weld¬ 
ments  and  high-cycle  fatigue  tests  in  air  and  seawater  showed  properties  equivalent  to 
HY-100  steel  in  every  case.  The  steels  showed  similar  fatigue  crack  growth  rate  properties. 
General  corrosion,  crevice  corrosion,  galvanic  corrosion,  and  high-velocity  seawater  parallel 
flow  and  cavitation  tests  of  HSLA-100  in  seawater  showed  that  the  corrosion  behavior  of 
HY-100  and  HSLA-100  steels  was  comparable. 
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Technology  Readiness  Level  6: 

Model/Prototype  Tests 

Representative  model  or  prototype  system,  which  is  well  beyond  the  bread¬ 
board  tested  at  TRL  5  and  is  tested  in  a  relevant  environment. 

The  evaluation  of  HSLA-100  steel  production  plates  concluded  that  the  mechanical 
properties  of  production  plate,  welding  and  weldability  screening  tests,  fatigue  properties, 
and  corrosion  properties  demonstrated  that  the  system  was  viable  for  certification  for  com¬ 
batant  ship  structure.  Evaluation  as  a  system  by  explosion  bulge  and  crack-starter  bulge 
tests,  fragment  penetration  resistance  tests,  and  ballistic  property  tests  was  demonstrated  in 
the  next  phase. 

Explosion  bulge  and  crack  starter  explosion  bulge  tests  of  2-inch  thick  weldments 
by  GMAW,  SMAW,  and  SAW  of  HSLA-100  steel  were  successfully  conducted.  The 
weldments  were  fabricated  within  the  recommended  preheat/interpass  temperatures  expected 
for  HSLA-100  fabrication,  exhibited  no  indications  of  hydrogen  damage,  and  passed  the 
explosion  bulge  test  requirements. 

In  1987,  NAVSEA  initiated  projects  at  Electric  Boat  Corporation  and  Newport 
News  Shipbuilding  (NNS)  to  evaluate  the  weldability  of  HSLA-100  steel  under  various 
preheat  conditions  in  a  production  environment.  The  results  of  the  weldability  evaluation 
demonstrated  that  HSLA-100  steel  could  be  welded  at  up  to  1.25 -inch  thick  at  60  °F  mini¬ 
mum  preheat,  with  the  same  processes  and  consumables  being  used  for  HY-80/100  steels. 

Based  on  NNS’  welding  and  weldability  evaluations  of  HSLA-100  using  HY-100 
welding  consumables,  welding  preheat/interpass  temperature  limits  were  established.  Pre¬ 
heat  was  recommended  for  SAW  and  SMAW,  based  on  the  weld  metal  cracking  tendencies 
noted  for  these  flux-assisted  processes  in  the  weldability  testing.  For  GMAW  and  SAW, 
difficulties  were  experienced  in  obtaining  MIL-100S-2  and  MIL-120S-2  wire  electrodes 
(low  hydrogen  content)  with  acceptable  wire-feed  characteristics  for  elimination  of  preheat 
for  heavy-gage  plate  welding.  Research  projects  are  in  progress  to  develop  welding  con¬ 
sumables  specifically  for  HSLA-100  to  achieve  preheat-free  welding  in  heavy  plate,  highly 
restrained  welds. 

Ballistic  evaluations  demonstrated  that  HSLA-100  steel  and  GMAW  (MIL-100S-1) 
weldments  (fabricated  without  preheat)  were  equivalent  to  HY-100  steel  and  weldments  in 
ballistic  resistance.  Both  steels  were  comparable  to  Army  Rolled  Homogeneous  Armor. 

NNS  completed  weld  qualification  and  weldability  testing  to  conduct  pulsed-arc 
GMAW  and  SAW  of  HSLA-100  in  thicknesses  greater  than  1  inch  through  1-5/8  inch  at 
60  °F  preheat  using  MIL-100S-2  electrode.  NAVSEA  approved  the  procedures.  It  should 
also  be  noted  that  Ingalls  Shipbuilding  Division  (ISD)  conducted  weld  qualification  and 
weldability  tests  of  HSLA-100  up  to  1-inch  gage  using  both  HY-100-  and  HY-80-type 
welding  consumables  and  processes. 

The  present  material  specification  for  HSLA-80  and  HSLA-100  steel  strip,  sheet, 
and  plate  is  MIL-S-24645A,  with  Amendment  1  of  24  September  1990.  HSLA-100  was 
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certified  by  NAVSEA  for  surface  ship  construction  in  thicknesses  up  to  4  inches,  13  March 

1989. 
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Technology  Readiness  Level  7: 
Prototype  Near  or  at  Planned  Operational  System 


TRL-7  is  a  major  step  from  TRL  6,  requiring  the  demonstration  of  an 
actual  prototype  in  an  operational  environment. 

The  fabrication  of  a  series  of  structural  performance  models  was  completed  under 
shipyard  welding  conditions.  Holding  bulkhead  panel  models,  foundation  models,  and  a 
full-scale  foundation  were  evaluated  and  demonstrated  satisfactory  structural  performance. 

The  Electric  Boat  Division  [General  Dynamics  Corporation]  fabricated  the  full-scale 
foundation  and  a  small,  heavy-gage  tank  model.  NNS  partially  completed  the  fabrication  of 
a  full-scale  hard  tank;  however,  a  funding  shortage  precluded  tests.  In  these  shipyard  fabri¬ 
cation  exercises,  all  weld  cracking  was  related  to  SMAW  and  SAW  consumables  (where 
cracking  occurred  even  when  HY-100  preheat  temperatures  were  used)  or  to  improper 
welding  practices.  No  HAZ  cracking  occurred  in  HSLA-100. 

Hydrostatic  tests  of  full-gage  bulkhead  panel  models  are  an  extreme  test  of  plating- 
to-stiffener  strength  and  HAZ  ductility.  The  HSLA-100  panel  models  exceeded  anticipated 
holding  pressure  levels,  withstanding  over  twice  the  holding  pressure  of  identical  HY-100 
panel  models.  A  series  of  foundation  beam  elements  (full-scale)  and  the  full-scale 
SSN  688-type  AC  foundation  were  installed  and  tested  on  a  floating  shock  platform.  The 
structures  were  subjected  to  a  series  of  underwater  explosion  (UNDEX)  shock  tests.  For  a 
series  of  3  UNDEX  events,  the  structural  response  of  the  HSLA-100  items  indicated  no 
cracking  or  excessive  deformation  in  any  structural  joint. 

Key  References: 

Czyryca,  E.J.,  Assessment  of  HSLA-80  and  HSLA-100  Steels  for  Submarine  Non-Pressure 
Hull  Applications,  CARDIVNSWC-TR-6 1-94/38,  Preliminary,  February  1995. 

Fugate,  S.P.,  Fabrication  of  the  HSLA-100  Foundation  Structure,  Electric  Boat  Division  of 
General  Dynamics  Corporation,  Report  No.  276,  Contract  No.  00024-86-C-2059, 
June  1988. 

Kenney,  D.P.,  and  S.P.  Fugate,  Fabrication  of  an  HSLA-100  Model  Structure,  Electric  Boat 
Division  Report  No.  PDE-279,  NAVSEA  Contract  No.  N00024-86-C-2059,  July  1989. 

Knight,  D.E.,  and  J.R.  Carlberg,  Shock  Performance  Evaluation  of  HSLA-100  Foundation 
Structures,  DTRC/SSPD-90- 172-1,  October  1989. 

Spaulding,  R.S.,  P.D.  Thomas,  and  R.A.  Spitzer,  HSLA-100  Hard  Tank  Fabrication  and 
Fatigue  Model,  Design  and  Fabrication  Report,  Newport  News  Shipbuilding  and  Drydock 
Company,  Report  G2001-  0059,  Contract  No.  N00024-87-C-2012,  9  September  1988. 

Thomas,  P.D.,  HSLA-100  Hard  Tank  Fabrication  and  Fatigue  Model  Construction,  New¬ 
port  News  Shipbuilding  and  Drydock  Company,  Report  No.  ES10-1,  Contract  No.  00024- 
85-C-2056,29  July  1988. 
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HSLA-100  Steel/LC-100  Weld  Metal 
Box-Tank  Fatigue  Model 
Overall  View  of  Model  Exterior/End  Hatch  Open 


HSLA-100  Holding  Bulkhead  Panel  Model: 

Before  Test  (Left)  and  After  Hydrostatic  Test  to  Rupture  (Right) 
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Technology  Readiness  Level  8: 

Technology  Demonstrated  In  Operation 

Technology  has  been  proven  to  work  in  its  final  form  and  under  expected 

conditions. 

In  1989,  NAVSEA  certified  HSLA- 100  steel  for  surface  ship  construction  in  thick¬ 
nesses  up  to  4  inches.  At  that  time,  the  USS  JOHN  C.  STENNIS  (CVN  74)  was  approved, 
indicating  that  HSLA-100  steel  was  a  qualified  substitute  for  HY-80/100  steel  in  CVN  con¬ 
struction.  Fabrication  was  to  be  conducted  in  accordance  with  MIL-STD-1689A(SH),  Fab¬ 
rication,  Welding,  and  Inspection  of  Ships  Structure.  The  experience  base  for  welding 
HSLA-100  steel  was  too  limited  to  allow  the  wholesale  substitution  for  all  HY-80/100  steel 
in  the  unrestricted  areas  of  the  carrier.  Therefore,  an  implementation  plan  for  incorporation 
was  submitted,  and  NAVSEA  approved  this  plan. 

The  CVN  74  main  deck  was  the  chosen  area  for  HSLA-100,  and  approximately 
770  LT  were  earmarked.  The  thicknesses  in  this  area  were  7/8-inch  and  1-inch  thick 
HSLA-100.  The  fabrication  results  were  excellent.  A  total  of  16,656  inches  of  butt  joints  in 
the  7/8-inch  plate  were  welded,  with  only  8  inches  requiring  repair.  In  the  1-inch  plate, 
16,524  inches  of  butt  joints  were  welded,  and  no  defects  were  found.  Since  the  ship  was 
under  construction  at  the  time  of  the  implementation  plan,  the  total  tonnage  inserted  into 
CVN  74  was  limited  to  1,250  LT,  mostly  above  main  deck. 

NNS  used  HSLA-100  steel  during  CVN  74  construction.  Approximately  700  tons 
of  HSLA-100  steel  plate  in  7/8-  and  1-inch  thicknesses  were  used  for  main  deck  panel 
assemblies  with  longitudinal  and  transverse  stiffeners  without  preheat  (65  to  80  °F  shop 
temperature).  One  hundred  percent  magnetic  particle  inspection  was  performed  on  all 
HSLA-100  butt  welds.  In  1,400  feet  of  7/8-inch  thick  HSLA-100  butt  weld  inspected  by 
MT,  only  2  repairs  (8  inches  total)  were  required,  not  related  to  hydrogen-type  defects.  The 
same  length  of  1-inch  thick  HSLA-100  butt  weld  inspected  by  magnetic  particle  inspection 
showed  no  defects.  A  total  of  1,250  tons  of  HSLA-100  were  used  in  CVN  74,  with  over 
4,000  feet  of  weldment  inspection  requiring  32  inches  total  repair  (less  than  0.01  percent). 

NNS  completed  weld  qualification  and  weldability  testing  to  conduct  pulsed-arc 
GMAW  and  SAW  of  HSLA-100  in  thicknesses  greater  than  1  inch  through  1-5/8  inch  at 
60  °F  preheat  using  MIL-100S-2  electrode.  NAVSEA  approved  the  procedures.  It  should 
also  be  noted  that  ISD  conducted  weld  qualification  and  weldability  tests  of  HSLA-100  up 
to  1-inch  gage  using  both  HY-100-  and  HY-80-type  welding  consumables  and  processes. 
The  flight  deck  of  the  USS  BATAAN  (LHD  5)  was  successfully  fabricated  with  HSLA-100 
plate  (in  place  of  HY-100  steel)  for  cost  savings,  as  were  subsequent  vessels  of  the  same 
class. 

Key  References: 

Christein,  J.P.  and  J.L.  Warren,  “Implementation  of  HSLA-100  Steel  in  Aircraft  Carrier 
Construction  -  CVN  74,”  Journal  of  Ship  Production ,  Society  of  Naval  Architects  and 
Marine  Engineers,  1994. 
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CVN  74  HSLA-100  Steel  Main  Deck  Panel  Fabrication 
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Technology  Readiness  Level  9: 

Implementation  of  the  Technology  in  Service 

Actual  application  of  the  technology  in  its  final  form  and  under  mission 

conditions. 

Because  of  the  experience  gained  on  CVN  74,  wholesale  changes  to  HSLA-100 
were  made  on  CVN  75.  Approximately,  10,500  LT  of  HSLA-100  were  inserted  into 
CVN  75.  Most  of  the  replacement  was  for  decks  and  bulkheads  and  some  built-up  stiffen¬ 
ers.  The  HSLA-100  stiffeners  were  short  spans  with  heavy  web/flange  members. 
HSLA-100  steel  was  selected  to  replace  HY-100  for  fabrication  cost  reduction,  and,  as  a 
consequence,  HSLA-100  steel  has  been  used  in  place  of  HY-100  in  the  construction  of 
USSJOHN  C.  ST  ENNIS  (CVN  74),  USS  HARRY  S.  TRUMAN  (CVN  75),  and 
USS  RONALD  REAGAN  (CVN  76). 

On  CVN  76,  NAVSEA  08  approved  the  substitution  of  HSLA-100  for  HY-80/100 
structures  outside  the  primary  shield  tank,  opening  another  area  for  substitution.  On 
CVN  77,  expended  use  of  HSLA-100  plate  continues.  NNS  expects  to  qualify  reduced  pre¬ 
heat  for  welding  up  to  2  inches,  adding  over  4,000  LT  of  HSLA-100  where  significant  fab¬ 
rication  cost  reduction  is  gained  over  HY-100  in  this  thickness  range.  Depending  on 
complexity  of  the  structure,  estimated  cost  savings,  for  HSLA-100  vs.  HY-100  fabrication  in 
CVN  74  construction  range  from  $500  to  $3,000  per  ton  of  fabricated  structure. 

The  table  below  summarizes  the  tonnage  of  HSLA-100  steel  plate  used  to  date  in 
construction  of  U.S.  Navy  combatant  ships.  The  continued  expansion  of  the  use  of 
HSLA-100  steel  is  planned  for  CVNX  (CVN  78)  design,  including  the  heavy  plating  and 
foundation  in  the  propulsion  area. 


Class 

Vessels 

LT 

CVN  68 

CVN  74 

2,080 

CVN  75 

11,600 

CVN  76 

12,500 

CVN  77 

12,500 

LHD  1 

LHD  5 

1,180 

LHD  6 

1,200 
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ACRONYMS  AND  ABBREVIATIONS  FOR  APPENDIX  D8 


ASM 

American  Society  for  Metals  International 

ASTM 

American  Society  for  Testing  and  Materials 

CG 

Carrier  Group 

CVN 

Aircraft  Carrier,  Nuclear 

CVNX 

Aircraft  Carrier,  Nuclear,  Experimental 

DDG 

Guided  Missile  Destroyer 

DTNSRDC 

David  Taylor  Naval  Ships  Research  and  Development  Center 

DTRC 

David  Taylor  Research  Center 

FFG 

Guided  Missile  Frigate 

GMAW-P 

Pulsed  Gas  Metal  Arc  Welding 

GMAW-S 

Short  Circuiting  Gas  Metal  Arc  Welding 

GMLRS 

Guided  Multiple  Launch  Rocket  System 

HAZ 

heat-affected  zone 

HSLA 

High-Strength  Low-Alloy 

HY 

High  Yield  Strength  (steel) 

IMU 

Inertial  measurement  Unit 

INCO 

International  Nickel  Company 

ISD 

Ingalls  Shipbuilding  Division 

LHD 

Amphibious  Assault  Ship 

LSD 

Dock  Landing  Ship 

LT 

long  ton 

NAVSEA 

Naval  Sea  Systems  Command 

NNS 

Newport  News  Shipbuilding 

OTC 

Offshore  Technology  Conference 

RLG 

Ring  Laser  Gyro 

SAW 

Submerged  Arc  Welding 

SMAW 

Shielded  Metal  Arc  Welding 

SME 

Society  for  Mining,  Metallurgy,  and  Exploration 

TM 

Technical  Manual 

TRL 

Technology  Readiness  Level 

UNDEX 

underwater  explosion 

8  These  acronyms  are  for  Appendix  D  (pp.  D-l  through  D-32). 
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APPENDIX  E 

SOFTWARE-SPECIFIC  DEFINITIONS  AND  DESCRIPTIONS  OF 
TECHNOLOGY  READINESS  LEVELS  (TRLs) 
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APPENDIX  E 

SOFTWARE-SPECIFIC  DEFINITIONS  AND  DESCRIPTIONS  OF 
TECHNOLOGY  READINESS  LEVELS  (TRLs) 


When  considering  the  maturity  of  developing  software,  U.  S.  Army  technologists 
determined  that  they  would  benefit  by  more  specific  and  directly  relevant  TRLs  than  the 
basic  ones  provided  in  the  original  Department  of  Defense  (DoD)  5000  series  documents. 
Accordingly,  these  Army  technologists  developed  a  software-appropriate  set  of  TRL 
definitions  and  descriptions,  which  are  provided  in  Table  E-l  for  reference  and  potential 
use. 


Table  E-1.  Software  TRL  Definitions 


TRL 

Definition 

Description 

1 

SW:  Functionality  conjectural 

Lowest  level  of  software  readiness.  Basic 
research  begins  to  be  translated  into  applied 
research  and  development.  Examples  might 
include  a  concept  that  can  be  implemented  in 
software  or  analytic  studies  of  an  algorithm’s 
basic  properties. 

2 

SW:  Technology  concept  and/or  application 
formulated 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be 
invented.  Applications  may  be  speculative  and 
there  may  be  no  proof  or  detailed  analysis  to 
support  the  assumptions.  Examples  are 
limited  to  analytic  studies. 

3 

SW:  Analytical  and  experimental  critical 
functions  and/or  characteristic  proof  of 
concept 

Active  research  and  development  is  initiated. 
This  includes  analytical  studies  to  produce 
code  that  validates  analytical  predictions  of 
separate  software  elements.  Examples 
include  software  components  that  are  not  yet 
integrated  or  representative  but  satisfy  an 
operational  need.  Algorithms  run  on  a 
surrogate  processor  in  a  laboratory 
environment. 
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Table  E-1.  Software  TRL  Definitions  (Continued) 


Definition 


SW:  Functionality  demonstrated  in  a 
laboratory  environment 


SW:  Functionality  and  performance 
demonstrated  in  a  relevant  environment 


SW:  Functionality  and  performance 
demonstrated  in  a  realistic  simulated 
(live/virtual)  operational  environment 


SW:  Functionality  and  performance 
demonstrated  in  an  operational  test 
environment. 


Description 


Basic  software  components  are  integrated  to 
establish  that  they  will  work  together.  They  are 
relatively  primitive  with  regard  to  efficiency 
and  reliability  compared  with  the  eventual 
system.  System  software  architecture 
development  initiated  to  include 
interoperability,  reliability,  maintainability, 
extensibility,  scalability,  and  security  issues. 
Software  integrated  with  simulated  current/ 
legacy  elements  as  appropriate. 


Reliability  of  software  ensemble  increases 
significantly.  The  basic  software  components 
are  integrated  with  reasonably  realistic 
supporting  elements  so  that  it  can  be  tested  in 
a  simulated  environment.  Examples  include 
“high-fidelity”  laboratory  integration  of 
software  components. 

System  software  architecture  established. 
Algorithms  run  on  a  processor(s)  with 
characteristics  expected  in  the  operational 
environment.  Software  releases  are  “Alpha” 
versions  and  configuration  control  initiated. 
Verification,  Validation,  and  Accreditation 
(VV&A)  initiated. 


Representative  model  or  prototype  system, 
which  is  well  beyond  that  of  TRL  5,  is  tested  in 
a  relevant  environment.  Represents  a  major 
step  up  in  software-demonstrated  readiness. 
Examples  include  testing  a  prototype  in  a 
live/virtual  experiment  or  in  simulated 
operational  environment.  Algorithm  run  on 
processor  or  operational  environment 
integrated  with  actual  external  entities. 
Software  releases  are  “Beta”  versions  and  are 
configuration  controlled.  Software  support 
structure  in  development.  VV&A  in  process. 


Represents  a  major  step  up  from  TRL  6, 
requiring  the  demonstration  of  an  actual 
system  prototype  in  an  operational 
environment,  such  as  in  a  command  post  or 
air/ground  vehicle.  Algorithms  run  on 
processor  of  the  operational  environment 
integrated  with  actual  external  entities. 
Software  support  structure  in  place.  Software 
releases  are  in  distinct  versions.  Frequency 
and  severity  of  software  deficiency  reports  do 
not  significantly  degrade  functionality  or 
performance.  VV&A  completed. 


Table  E-1.  Software  TRL  Definitions  (Continued) 


TRL 

Definition 

Description 

8 

SW:  Functionality,  performance,  and  quality 
attributes  validated  in  an  operational 
environment. 

Software  has  been  demonstrated  to  work  in  its 
final  form  and  under  expected  conditions.  In 
most  cases,  this  TRL  represents  the  end  of 
system  development.  Examples  include  test 
and  evaluation  of  the  software  in  its  intended 
system  to  determine  if  it  meets  design 
specifications.  Software  releases  are 
production  versions  and  are  configuration 
controlled  in  a  secure  environment.  Software 
deficiencies  are  rapidly  resolved  through 
support  structure. 

9 

SW:  Functionality,  performance  and  quality 
attributes  proven  in  an  operational 
environment  through  successive  successful 
accomplishment  of  mission  operations. 

Actual  application  of  the  software  in  its  final 
form  and  under  mission  conditions,  such  as 
those  encountered  in  operational  test  and 
evaluation.  In  almost  all  cases,  this  is  the  end 
of  the  last  “bug  fixing”  aspects  of  system 
development.  Examples  include  using  the 
system  under  operational  mission  conditions. 
Software  releases  are  production  versions  and 
are  configuration  controlled.  Frequency  and 
severity  of  software  deficiencies  are  at  a 
minimum 
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ACRONYMS  AND  ABBREVIATIONS  FOR  APPENDIX  E 

DoD 
TRL 
VV&A 


Department  of  Defense 
Technology  Readiness  Level 
Verification,  Validation,  and  Accreditation 
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APPENDIX  F 

BIOMEDICAL  TECHNOLOGY  READINESS  LEVELS  (TRLs) 


Medical-related  items  require  TRL  definitions  and  descriptions  appropriate  to  the 
technologies  upon  which  they  are  based  and  that  account  for  the  statutes  and  regulations 
that  govern  their  development  and  use.  In  recognition  of  these  factors,  the  U.S.  Army 
Medical  R&D  Command  took  the  initiative  to  establish  appropriate  definitions,  descriptions, 
and  processes  in  the  context  of  military  medical  R&D  and  the  statutory  and  regulatory 
requirements  under  the  stewardship  of  the  Food  and  Drug  Administration  (FDA).  The 
excellent  result  of  their  effort  is  provided  in  this  appendix,  with  slight  editing  of  presentation 
(but  not  substance)  to  suit  this  deskbook. 
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— Technology 
—  Readiness 
-  Levels  (TRLs) 


A.  BACKGROUND 

Department  of  Defense  (DoD)  policy  mandates  the  use  of  U.S.  Food  and  Drug  Adminis¬ 
tration  (FDA)-approved  products  for  force  health  protection,12  and  the  U.S.  Army 
Medical  Research  and  Materiel  Command  (USAMRMC)  has  always  adhered  to  the 
regulatory  requirements  of  the  FDA  regarding  its  studies  of  drugs,  biologies,  and  devices 
in  humans.  To  ensure  compliance  with  the  clinical  phases  of  the  FDA-regulated  process 
and  to  reduce  technological  risk,  the  USAMRMC  developed  and  recently  updated  their 
general  guidelines  for  assigning  TRLs  to  drug,  vaccine,  and  medical  device  development 
programs.13 

These  guidelines  are  not  considered  absolutes,  and  characterization  of  activities  associ¬ 
ated  with  TRLs  can  and  does  vary  at  times.  The  science  and  technology  (S&T)  and 
acquisition  program  managers  (PMs)  work  together  in  exercising  discretion  in  the  selec¬ 
tion,  progression,  and  timing  of  specific  activities  to  be  accomplished  in  the  attainment  of 
particular  TRLs.  Such  flexibility  and  tailoring  are  needed  to  align  the  TRL  decision  crite¬ 
ria  appropriately  with  the  maturation  and  risk  characteristics  of  a  particular  technology, 
including  consideration  of  the  associated  investment  strategy  and  transition  procedures 
that  may  vary  among  PMs. 

The  lower  a  critical  technology’s  TRL  when  transitioning  from  technology  development 
to  product  development,  the  greater  the  risks.  For  medical  technologies,  risk  reduction  is 
not  linear  across  TRLs.  The  rate  of  risk  reduction  remains  very  low  until  very  late.  His¬ 
torically,  FDA-regulated  products,  such  as  vaccines,  do  not  achieve  significant  risk 
reduction  (i.e.,  greater  than  50  percent)  until  completion  of  Phase  3  clinical  trials  and 
approval  of  a  biologies  license  application  by  the  FDA  (TRL  8).  Industry  experience  is 
that  only  one  in  four  vaccines  going  into  Phase  3  trials  is  licensed.  Similarly,  whereas 
technology  maturation  is  commonly  perceived  as  a  sequential  continuum  of  activities 
from  basic  research,  through  development,  to  production  and  deployment,  the  evolution 
of  the  TRL  for  a  critical  technology  may  not  be  sequential,  especially  in  those  cases 

12  For  example,  Department  of  Defense  Directive  (DoDD)  6200.2,  August  1,  2000,  or  Health  Affairs  Policy  95-011, 
July  26,  1995. 

13  Biomedical  Technolog y  Readiness  Levels  (TRLs),  prepared  for  the  Commander,  U.S.  Army  Medical  Research  and 
Materiel  Command,  under  Contract  number  DAMD17-98-D-0022,  Science  Applications  International  Corporation, 

3  June  2003. 
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where  FDA  anchors  are  undefined.  In  cases  of  success  or  failure,  the  incremental  change 
in  the  level  of  technology  readiness  may  be  greater  than  a  single  TRL.  For  example,  upon 
successful  completion  of  a  pivotal  study,  biomedical  information  technology  readiness 
may  move  from  TRL  3  or  4  to  TRL  9. 

Biomedical  TRL  descriptions  provide  a  systematic  way  for  the  S&T  community  to  assess 
and  communicate  to  the  MDA  the  level  of  maturity  of  a  particular  technology  or  combi¬ 
nation  of  technologies  as  they  relate  to  the  particular  category  and  the  maturity  necessary 
for  successful  product  development.  This  appendix  provides  equivalent  TRL  descriptions 
applicable  to  biomedical  technologies  in  four  categories: 

1.  Pharmaceutical  (i.e.,  drugs) 

2.  Pharmaceutical  (i.e.,  biologics/vaccines) 

3.  Medical  Devices 

4.  Medical  Information  Management/Information  Technology  (IM/IT)  and  Medical 
Informatics. 

The  TRLs  for  the  first  three  categories  have  been  developed  from  the  DoD’s  generic 
definitions,  the  applicable  FDA  regulatory  process,  and  industry  practices  and  experience 
with  its  research  and  development  (R&D)  processes  (discovery  through  manufacturing, 
production,  and  marketing).  The  last  category  includes  elements  of  formal  regulatory 
processes  and  logical  events  in  deriving  comparable  levels  of  maturity.  The  USAMRMC 
intends  to  use  external  anchors  such  as  “FDA  events”  wherever  practical  to  define  each 
TRL  decision  criterion.  Furthermore,  activities  described  as  occurring  between  succes¬ 
sive  TRL  decision  criteria  are  intended  to  exemplify  the  kinds  of  activities  that  routinely 
take  place  when  maturation  is  sequential  and  stepwise.  However,  these  examples  are 
neither  mandatory  nor  all-inclusive. 

Figure  1-1  and  Table  1-1  build  upon  this  work  by  providing  examples  of  supporting 
information  and  documentation  required  to  support  the  assignment  of  TRLs  as  the  pro¬ 
gram  progresses. 

In  addition,  a  description  of  the  FDA  regulatory  process,  points  of  contact  (POCs)  within 
FDA,  sources  of  additional  information,  a  glossary  of  terms,  and  a  list  of  acronyms  and 
abbreviations  are  provided. 


The  proponent  for  this  document  is  the  Deputy  for  Research  and  Development: 

Commander,  U.S.  Army  Medical  Research  and  Materiel  Command 


ATTN:  MCMR-ZC 
504  Scott  Street 


Fort  Detrick,  MD  21702-5012 
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Interim  Defense  Acquisition  Guidebook  ( Interim  Guidebook),  dated  October  30,  2002.  This  Interim  Guidebook  (formerly  the  DoD  5000. 2-R,  dated  April  5,  2002) 
contains  nonmandatory  guidance  on  best  practices,  lessons  learned,  and  expectations.  It  is  anticipated  that  the  Interim  Guidebook  will  be  revised  in  the  near  future. 
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Table  1-1.  Proposed  TRLs  for  Medical  (RDT&E)  (Continued) 
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TRL  5  NASA /Interim  Guidebook  TRL  Definition:  Component  and/or  breadboard  validation  in  a  relevant  environment  (Continued) 
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nary  claim  that  the  medical  device  appears 
substantially  equivalent  to  a  predicate  device, 
the  proposed  classification  is  consistent  with 
21CFR860,  there  is  a  functioning  model,  and 
testing  results  support  substantial  equivalency. 


F-14 


F-15 


F-16 


Table  1-1.  Proposed  TRLs  for  Medical  (RDT&E)  (Continued) 
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B.  THE  FDA  REGULATORY  PROCESS 


The  FDA  regulates  products  to  protect  the  public  health  by  ensuring  that  human  pharmaceu¬ 
ticals  (drugs  and  biologics/vaccines)  are  safe  and  effective  and  that  there  is  reasonable  assur¬ 
ance  of  the  safety  and  effectiveness  of  medical  devices  intended  for  human  use  in  the  United 
States.  Three  FDA  centers  are  charged  with  this  mission: 

1.  The  FDA  Center  for  Drug  Evaluation  and  Research  (CDER).  CDER  regulates 
drugs  and  some  biologic  products  (antibodies,  cytokines,  growth  factors,  enzymes,  and 
proteins  extracted  from  animals  or  microorganisms). 

2.  The  FDA  Center  for  Biologies  Evaluation  and  Research  (CBER).  CBER  regulates 
vaccines,  blood  and  plasma  products,  viral- vectored  gene  therapy,  products  composed 
of  human  or  animal  cells,  antitoxins,  and  select  in  vitro  diagnostics.  CBER  also  holds 
regulatory  authority  over  Human  Immunodeficiency  Virus  (HIV)  test  kits  and  medical 
devices  involved  in  collecting,  processing,  testing,  manufacturing,  and  administering 
blood  products. 

3.  The  FDA  Center  for  Devices  and  Radiological  Health  (CDRH).  CDRH  is  respon¬ 
sible  for  regulating  manufactured,  repackaged,  relabeled,  and/or  imported  medical 
devices  that  are  sold  in  the  United  States  (except  those  devices  regulated  by  CBER). 

1.  Pharmaceuticals 

Drugs  and  biologics/vaccines  follow  parallel  developmental  regulatory  pathways  (see 
Table  1-1).  During  preclinical  development,  the  sponsor  evaluates  the  toxicology  and  pharma¬ 
cology  of  the  new  drug  or  biologic  through  in  vitro  and  animal  testing.  Preclinical  test  results 
and  any  available  past  human  experiences  of  the  drug  or  biologic  are  incorporated  in  an 
Investigational  New  Drug  (IND)  application  and  submitted  to  the  FDA  for  review.  If  no  safety 
issues  are  found,  human  clinical  testing  of  the  new  drug  or  biologic  can  be  initiated  after  30 
days.  Clinical  testing  proceeds  in  three  successive  phases,  starting  with  a  small  group  of 
human  subjects  (Phase  1)  and  progressing  to  a  larger  population  of  human  subjects  (Phase  3). 
Only  by  qualified  investigators,  selected  by  the  sponsor,  and  in  accordance  with  Good  Clinical 
Practice  (GCP)  [21CFR312.53  and  21CFR312.62  Federal  Register  25692],  conduct  clinical 
trials.  The  safety  and  effectiveness  results  of  clinical  testing  comprise  the  most  important 
factor  in  the  approval  or  disapproval  of  the  new  drug  or  biologic.  All  active  INDs  require 
submission  of  an  annual  IND  report  to  the  FDA.  The  results  of  the  human  clinical  tests  and  all 
chemistry  and  manufacturing  information  are  submitted  either  in  a  New  Drug  Application 
(NDA)  for  drug  products  or  a  Biologies  License  Application  (BLA)  for  biologic  products.  The 
appropriate  FDA  center  reviews  the  NDA  or  BLA,  and,  upon  approval,  the  drug  or  biologic 
product  can  be  entered  into  interstate  commerce  or  marketed  in  the  United  States.  EDA 
approval  is  for  the  specific  indication(s)  identified  in  the  marketing  application.  Additional  or 
modified  medical  indications  require  the  submission  of  an  amendment  or  a  new  marketing 
application.  A  new  marketing  application  may  require  additional  human  clinical  data  acquired 
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through  IND  regulations.  With  some  new  drugs  or  biologics/vaccines,  the  FDA  may  require 
additional  reporting  requirements  after  approval,  termed  Phase  4  or  postmarketing  surveil¬ 
lance.  Manufacturers  are  required  to  track  and  report  the  number  and  severity  of  adverse 
events  attributable  to  each  product  for  a  specified  time  period.  Severe  adverse  events  detected 
during  postapproval  can  lead  to  a  product  recall  or  mandatory  withdrawal  from  the  market. 
All  drugs  and  biologics/vaccines  must  comply  with  current  Good  Manufacturing  Practice 
(cGMP)  and  labeling  regulations. 

With  certain  drug  or  biologic  products,  human  clinical  studies  are  not  ethical  or  feasible 
because  the  studies  would  involve  administering  a  potentially  lethal  or  permanently  disabling 
toxic  substance  or  organism  to  healthy  human  volunteers.  In  2002,  the  FDA  addressed  this 
issue  with  new  regulations  that  allow  for  the  approval  of  new  drug  and  biologic  products 
based  on  evidence  of  effectiveness  in  animals  [21CFR314  and  21CFR601].  In  February  2003 
under  the  new  federal  regulations,  DoD  was  able  to  gain  approval  of  pyridostigmine  bromide 
for  prophylaxis  against  the  lethal  effects  of  the  soman  nerve  agent. 

2.  Medical  Devices 

The  FDA  CDRH  regulates  most  medical  devices,  and  they  have  classified  each  device  in  the 
Code  of  Federal  Regulations  (CFR).  Classification  of  devices  into  one  of  three  classes  is 
based  on  the  level  of  regulatory  control  that  is  necessary  to  ensure  the  safety  and  effectiveness 
of  a  medical  device,  with  Class  I  and  Class  III  devices  being  the  least  and  most  regulated, 
respectively.  The  sponsor  normally  proposes  the  classification  level  of  a  device  using 
21CFR860  as  a  guide.  Most  importantly,  the  classification  of  the  device  will  identify,  unless 
exempt  (e.g.,  most  of  the  Class  I  devices),  the  marketing  process  [either  premarket  notification 
(510(k))  or  premarket  approval  (PMA)]  that  the  manufacturer  must  complete  to  obtain  FDA 
clearance/approval  for  marketing.  All  classified  medical  devices  are  subject  to  cGMPs  and 
labeling  requirements.  An  approved  510(k)  or  PMA  allow  an  applicant  to  market  a  particular 
device  for  its  intended  purpose. 

The  FDA  approves  most  medical  devices  for  marketing  in  the  United  States  through  a  pre¬ 
market  notification  (510(k)).  The  applicant  must  show  that  the  new  device  is  substantially 
equivalent  to  one  or  more  predicate  devices  legally  marketed  in  the  United  States.  A  descrip¬ 
tion  of  all  tests  conducted  and  the  results  obtained  must  be  provided  in  sufficient  detail  to 
allow  the  FDA  to  determine  substantial  equivalence.  If  the  medical  device  is  found  to  be  sub¬ 
stantially  equivalent,  the  FDA  will  send  the  manufacturer  a  “substantially  equivalent  letter”  to 
clear  the  device  for  marketing.  If  the  FDA  finds  the  device  not  to  be  substantially  equivalent, 
the  FDA  sends  the  manufacturer  a  “not  substantially  equivalent  letter,”  and  the  device  cannot 
be  marketed.  At  this  point,  the  manufacturer  can  submit  another  510(k)  with  new  and/or  addi¬ 
tional  information  to  support  substantial  equivalence,  or  the  manufacturer  may  be  required  to 
submit  a  PMA. 

To  allow  a  Class  III  medical  device  (devices  are  those  that  support  or  sustain  human  life  or 
present  a  potential  risk  of  serious  illness  or  injury)  into  interstate  commerce  or  marketing,  a 
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PMA  is  required.  A  PMA  is  the  most  stringent  regulatory  submission  for  medical  devices. 
Class  III  devices  follow  somewhat  different  development  and  regulatory  paths  compared  with 
those  for  drugs  and  biologics/vaccines  (see  Table  1-1).  For  example,  if  human  clinical  infor¬ 
mation  is  required  to  establish  safety  and  efficacy,  the  regulatory  application  used  to  allow 
human  clinical  trials  is  called  an  Investigational  Device  Exemption  (IDE).  Approval  of  an 
IDE  allows  the  initiation  of  human  clinical  trials  of  an  investigational  device.  Qualified  prin¬ 
cipal  investigators  (Pis),  selected  by  the  sponsor  in  accordance  with  21CFR812.43,  conduct 
clinical  trials.  All  active  IDEs  require  submission  of  an  annual  report  to  the  FDA.  Safety  and 
efficacy  information  acquired  during  the  IDE  process  is  used  to  support  the  submission  of  a 
PMA,  and  the  FDA  must  approve  the  PMA  before  the  device  can  be  marketed.  As  with  drugs 
and  biologics/vaccines,  the  FDA  may  mandate  a  period  of  postmarketing  surveillance  during 
which  device-related  adverse  events  must  be  tracked  and  reported. 

C.  POCs 

FDA  Center  for  Devices  and  Radiological  Health  (CDRH) 

Web  site:  http://www.fda.gov/cdrh/ 

Questions  related  to  DoD  or  military: 

Ronald  Parr:  ipp@cdrh.fda.gov,  301-443-6597,  ext.  109 
Thomas  Cardamone:  tec@cdrh.fda.gov,  301-443-0806,  ext.  115 
Questions  can  also  be  addressed  to: 

Division  of  Small  Manufacturers,  International  and  Consumer  Assistance  (DSMICA) 
dsmica@cdrh.fda.gov 

FDA  Center  for  Drug  Evaluation  and  Research  (CDER) 

Procedures,  forms,  policies,  guidance  documents,  and  regulations  related  to  the  drug  approval 
process  may  be  accessed  from  the  Web  site: 
http://www.fda.gov/cder/regulatory/applications/default.htm 

Questions  related  to  DoD  or  military: 

Division  of  Counter-Terrorism  (HFD-970) 

CDER 

Voice:  301-827-7709 
Fax:  301-827-7722 
Questions  can  also  be  addressed  to: 

CDER  Division  of  Drug  Information 

301-827-4573 

druginfo@cder.fda.gov 
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FDA  Center  for  Biologies  Evaluation  and  Research  (CBER) 

Web  site:  http://www.fda.gov/cber/index.html 
Questions  may  be  addressed  to: 

CBER  Manufacturers  Assistance  and  Technical  Training  Branch 

800-835-4709  or  301-827-1800 

MATT@cber.fda.gov 

D.  ADDITIONAL  INFORMATION 

Federal  Food,  Drug,  and  Cosmetic  Act  (FD&C  Act) 

United  States  Code,  Title  21  -  Food  and  Drugs  (21USC) 

Chapter  9:  Federal  Food,  Drug,  and  Cosmetic  Act 
http://www.access.gpo.gov/uscode/title21/chapter9  .html 

FDA  Regulations 

Code  of  Federal  Regulations  (CFR):  Title  21  -  Food  and  Drugs  (21CFR) 
http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm  or 

http://www.access.gpo.gov/nara/cfr/index.html 

Drug  Approval 

The  CDER  Handbook',  http://www.fda.gov/cder/handbook/ 

CDERLearn:  http://www.fda.gov/cder/learn/CDERFearn/default.htm 

Medical  Device  Approval 

Device  Advice :  http://www.fda.gov/cdrh/devadvice/index.html 

Laws  Enforced  by  the  FDA: 

http://www.fda.gov/opacom/laws/lawtoc.htm 

Protection  of  Human  Subjects: 

32CFR219-  Protection  of  Human  Subjects  (also  referred  to  as  the  “Common  Rule”) 
(http://www.access.gpo.gov/nara/cfr/waisidx  02/32cfr219  02.html) 

DoDD  3216.2  (March  25,  2002)  Protection  of  Human  Subjects  and  Adherence  to  Ethical 
Standards  in  DoD-Supported  Research 

http://www.dtic.mil/whs/directives/corres/pdf/d32162  032502Zd32162p.pdf 
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GLOSSARY15 


Approval  Letter:  A  written  communication  to  an  applicant  from  the  Food  and  Drug  Admini¬ 
stration  (FDA)  approving  an  application  or  an  abbreviated  application  to  market  a  drug. 
[21CFR314.3] 

Approval  Order:  A  written  communication  to  an  applicant  from  the  FDA  approving  a  Pre¬ 
market  Approval  (PMA)  for  a  Medical  Devices  application.  [21CFR814.44] 

Biologic  or  Biological  Product:  Any  virus,  therapeutic  serum,  toxin,  antitoxin,  or  analogous 
product  applicable  to  the  prevention,  treatment,  or  cure  of  diseases  or  injuries  of  man. 
[21CFR600.3] 

Biologies  License  Application  (BLA):  An  application  to  the  FDA  for  approval  to  market  a 
biological  product.  [21CFR601.12] 

current  Good  Manufacturing  Practices  (cGMP):  Regulations  that  cover  the  methods  used 
in  and  the  facilities  and  controls  used  for  the  design,  manufacture,  packaging,  storage,  and 
installation  of  devices.  [21CFR820] 

Class  (Device):  One  of  the  three  categories  of  regulatory  control  for  medical  devices. 
[21CFR860.3] 

Class  I  Device:  The  class  of  devices  for  which  general  controls  are  sufficient  to  provide  rea¬ 
sonable  assurance  of  the  safety  and  effectiveness  of  the  device.  In  the  absence  of  sufficient 
information  to  make  that  determination,  the  device  is  not  life  supporting  and  does  not  present 
a  potential  unreasonable  risk  of  illness  or  injury.  [21CFR860.3] 

Class  II  Device:  The  class  of  devices  for  which  general  controls  alone  are  insufficient  to  pro¬ 
vide  reasonable  assurance  of  its  safety  and  effectiveness  and  for  which  there  is  sufficient 
information  to  establish  special  controls,  including  the  promulgation  of  performance  stan¬ 
dards.  For  a  device  that  is  purported  to  be  for  use  in  supporting  human  life,  the  Commissioner 
(FDA)  shall  examine  and  identify  the  special  controls,  if  any,  that  are  necessary  to  provide 
adequate  assurance  of  safety  and  effectiveness.  [21CFR860.3] 

Class  III  Device:  The  class  of  devices  for  which  premarket  approval  is  or  will  be  required.  A 
device  is  in  Class  III  if  insufficient  information  exists  to  determine  that  general  controls  are 
sufficient  to  provide  reasonable  assurance  of  its  safety  and  effectiveness,  if  the  device  is  life 
supporting,  or  if  the  device  presents  a  potential  unreasonable  risk  of  illness  or  injury. 
[21CFR860.3] 


15  Complete  definitions  and  explanations  of  terms  can  be  found  in  the  source  cited  in  brackets.  CFR  is  an 
acronym  for  the  Code  of  Federal  Regulations. 
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Classification  Name:  The  term  used  by  the  FDA  and  its  classification  panels  to  describe  a 
device  or  class  of  devices  for  purposes  of  classifying  devices  under  section  513  of  the  Federal 
Food,  Drug,  and  Cosmetic  (FD&C)  Act.  [21CFR807.3].  Approximately  1,700  different 
generic  types  of  devices  are  grouped  into  16  medical  specialties.  [21CFR862-892] 

Clinical  Hold:  An  FDA  order  to  delay  proposed  clinical  investigation  or  to  suspend  an 
ongoing  investigation. 

Clinical  Investigation:  Any  experiment  in  which  a  drug  that  involves  one  or  more  human 
subjects  is  administered,  dispensed  to,  or  used.  For  this  part,  an  experiment  is  any  use  of  a 
drug  except  for  the  use  of  a  marketed  drug  in  the  course  of  medical  practice.  [21CFR312.3] 

Clinical  Trial/Study:  Any  investigation  in  human  subjects  intended  to  discover  or  verify  the 
clinical,  pharmacological,  and/or  other  pharmacodynamic  effects  of  an  investigational  prod¬ 
uct/s),  and/or  identify  any  adverse  reactions  to  an  investigational  product(s),  and/or  study 
absorption,  distribution,  metabolism,  and  excretion  of  an  investigational  product(s)  with  the 
object  of  ascertaining  its  safety  and/or  efficacy.  The  terms  clinical  trial  and  clinical  study  are 
synonymous.  [62FR25692] 

Cosmetic:  (1)  Articles  intended  to  be  rubbed,  poured,  sprinkled  or  sprayed  on,  introduced  into 
or  otherwise  applied  to  the  human  body  or  any  part  thereof  for  cleansing,  beautifying,  pro¬ 
moting  attractiveness,  or  altering  appearance  and  (2)  articles  intended  for  use  as  a  component 
of  any  such  article.  This  term  shall  not  include  soap. 

Device  Master  Record  (DMR):  A  compilation  of  records  containing  the  procedures  and 
specifications  for  a  finished  device.  [21CFR820.3] 

Drug  or  Drug  Substance:  An  active  ingredient  that  is  intended  to  furnish  pharmacological 
activity  or  other  direct  effect  in  the  diagnosis,  cure,  mitigation,  treatment,  or  prevention  of 
disease  or  to  affect  the  structure  or  any  function  of  the  human  body.  [21CFR314.3] 

Drug  Product:  A  finished  dosage  form  (e.g.,  tablet,  capsule,  or  solution)  that  contains  a  drug 
substance,  generally,  but  not  necessarily,  in  association  with  one  or  more  other  ingredients. 
[21CFR314.3] 

FD&C  Act:  The  Federal  Food,  Drug,  and  Cosmetic  Act  [21USC301-397] 

FDA- Approved:  FDA  designation  given  to  drugs,  biologies,  and  medical  devices  that  have 
approved  marketing  applications.  Additional  or  modified  medical  indications  for  use  require 
submission  of  an  amendment  or  a  new  marketing  application.  A  new  marketing  application 
may  require  additional  human  clinical  data  acquired  through  Investigational  New  Drug  (IND) 
regulations. 
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General  Controls:  The  baseline  requirements  of  the  FD&C  Act  that  apply  to  all  medical 
devices.  In  addition  to  prohibiting  adulteration,  misbranding,  and  banned  devices,  the  general 
controls  contain  requirements  for  device  manufacturers.  These  requirements  include  device 
listing,  proper  labeling,  (manufacturing)  establishment  registration,  and  premarket  notification 
[510(k)]. 

Good  Clinical  Practice  (GCP):  A  standard  for  the  design,  conduct,  performance,  monitoring, 
auditing,  recording,  analyses,  and  reporting  of  clinical  trials.  It  provides  assurance  that  the 
data  and  reported  results  are  credible  and  accurate  and  that  the  rights,  integrity,  and  confiden¬ 
tiality  of  trial  subjects  are  protected.  [62FR25692] 

Good  Laboratory  Practice  (GLP):  Practices  for  conducting  nonclinical  laboratory  studies 
that  support  or  are  intended  to  support  applications  for  research  or  marketing  permits  for 
products  regulated  by  the  FDA.  [21CFR58.1] 

Investigational  Device  Exemption  (IDE):  Allows  the  investigational  device  to  be  used  in  a 
clinical  study  to  collect  safety  and  effectiveness  data  required  to  support  a  PMA  application  or 
a  Premarket  Notification  [5 10(k)]  submission  to  the  FDA.  [21CFR50,  56,  812] 

Investigational  New  Drug  (IND):  A  new  drug  or  biological  that  is  used  in  a  clinical  investi¬ 
gation.  The  term  also  includes  a  biological  product  that  is  used  in  vitro  for  diagnostic  pur¬ 
poses.  [21CFR312.3] 

IND  Application  [21CFR312.3]:  Allows  a  pharmaceutical  (drug/biologic)  to  be  used  in  a 
study  under  carefully  controlled  and  intensely  monitored  conditions  in  order  to  collect  safety 
and  effectiveness  data  required  to  support  a  New  Drug  Application  (NDA)  or  BLA. 

Investigator:  A  person  responsible  for  the  conduct  of  the  clinical  trial  at  a  trial  site.  If  a  trial 
is  conducted  by  a  team  of  individuals  at  a  trial  site,  the  investigator  is  the  responsible  leader  of 
the  team  and  may  be  called  the  principal  investigator  (or  PI).  [62FR25692] 

Label:  Any  display  of  written,  printed,  or  graphic  matter  on  the  immediate  container  or  pack¬ 
age  of,  or  affixed  to  any  article. 

Labeling:  Any  written,  printed  or  graphic  matter  accompanying  an  article  at  any  time  while 
such  article  is  in  interstate  commerce  or  held  for  sale  after  shipment  in  interstate  commerce. 
This  includes  manuals,  brochures,  advertising,  and  so  forth. 

License:  The  terminology  used  for  FDA’s  approval  to  market  a  biological  pharmaceutical  for 
a  given  set  of  indications  (see  also  FDA  Approved). 

Life-Supporting  or  Life- Sustaining  Device:  A  device  that  is  essential  to  or  that  yields 
information  that  is  essential  to  the  restoration  or  continuation  of  a  bodily  function  important  to 
the  continuation  of  human  life.  [21CFR860.3] 
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Medical  Device:  An  instrument,  apparatus,  implement,  machine,  contrivance,  implant,  in 
vitro  reagent,  or  other  similar  or  related  article,  including  any  component,  part,  or  accessory 
that  is 

•  Recognized  in  the  official  National  Formulary  or  U.S.  Pharmacopoeia  or  any  supple¬ 
ment  to  them 

•  Intended  for  use  in  diagnosing  disease  or  other  conditions  or  in  curing,  mitigating, 
treating,  or  preventing  disease  in  man  or  other  animals 

•  Intended  to  affect  the  structure  or  any  function  of  the  body  of  man  or  other  animals 
and  does  not  achieve  any  of  its  primary  intended  purposes  through  chemical  action 
within  or  on  the  body  of  man  or  other  animals  and  is  not  dependent  upon  being 
metabolized  for  achievement  of  any  of  its  primary  intended  purposes  [Section  201(h) 
of  the  FD&C  Act]. 

New  Drug  Application  (NDA):  An  application  to  the  FDA  for  approval  to  market  a  new 
drug.  [21CFR314.50] 

Preapproval  Inspection  (PAI):  An  FDA  inspection  of  a  facility  to 

•  Verify  the  integrity  (truthfulness,  accuracy,  and  completeness)  of  data  submitted  in 
support  of  an  application 

•  Evaluate  the  manufacturing  controls  for  the  preapproval  batches  upon  which  the  appli¬ 
cation  is  based  to  be  certain  that  the  company  can  actually  meet  the  commitments  in 
the  chemistry,  manufacturing,  and  controls  (CMC)  section  of  the  application 

•  Evaluate  the  capability  of  the  manufacturer  to  comply  with  GMPs 

•  Collect  samples  for  analysis. 

Postmarketing  Surveillance:  Tracking  and  reporting  the  number  and  severity  of  adverse 
events  attributable  to  each  product.  This  may  be  a  requirement  for  licensure  for  a  defined 
period  of  time  following  licensure. 

Premarket  Approval  for  Medical  Devices  (PMA):  Because  of  the  level  of  risk  associated 
with  Class  III  devices,  an  applicant  must  receive  FDA  approval  of  its  PMA  application  before 
marketing  the  device.  PMA  approval  is  based  on  a  determination  by  FDA  that  the  PMA 
contains  sufficient  valid  scientific  evidence  to  ensure  that  the  device  is  safe  and  effective  for 
its  intended  use(s).  [21CFR814] 
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Premarket  Notification  510(k):  An  application  submitted  to  the  FDA  to  demonstrate  that  a 
device  is  substantially  equivalent  [see  21USC513(I)(1)(A)]  to  a  device  that  is  legally  in  com¬ 
mercial  distribution  in  the  United  States  before  May  28,  1976,  or  to  a  device  that  has  been 
determined  by  FDA  to  be  substantially  equivalent.  [21CFR807.81] 

Quality  System  Inspection  Technique  (QSIT):  An  FDA  inspection  technique  that  focuses 
on  the  first  four  elements  of  the  seven  inspectional  subsets  of  the  Quality  System  Regulation 

(QSR). 

Quality  System  Regulation  (QSR):  The  1996  rewrite  of  the  device  section  of  the  cGMPs. 
[21CFR820] 

Serious  Adverse  Event  (SAE)  or  Serious  Adverse  Drug  Reaction  (ADR):  Any  untoward 
medical  occurrence  that  at  any  dose 

•  Results  in  death 

•  Is  life  threatening 

•  Requires  inpatient  hospitalization  or  prolongation  of  existing  hospitalization 

•  Results  in  persistent  or  significant  disability /incapacity 

•  Causes  a  congenital  anomaly/birth  defect. 

[See  the  International  Conference  on  Harmonisation  (ICH)  guidance  for  Clinical  Safety  Data 
Management:  Definitions  and  Standards  for  Expedited  Reporting].  [62FR25692] 

Special  Controls:  Class  II  devices  include  any  device  for  which  reasonable  assurance  of 
safety  and  effectiveness  can  be  obtained  by  applying  “special  controls.”  Special  controls  can 
include  special  labeling  requirements,  mandatory  performance  standards,  patient  registries, 
and  postmarket  surveillance. 

Sponsor:  An  individual,  company,  institution,  or  organization  that  takes  responsibility  for  the 
initiation,  management,  and/or  financing  of  a  clinical  trial.  [62FR25692] 

Subject:  A  human  who  participates  in  an  investigation,  either  as  a  recipient  of  the  IND  or  as  a 
control.  [21CFR312.3] 

Substantial  Equivalence  (SE):  A  device  is  substantially  equivalent  if,  in  comparison  to  a 
legally  marketed  device,  it  has  the  same  intended  use  as  a  predicate  and  has  the  same  techno¬ 
logical  characteristics  as  the  predicate  device.  SE  does  not  mean  the  devices  are  identical. 
[21CFR807.87] 

Type  B  Meeting:  Type  B  meetings  are  (1)  pre-IND  meetings  (21CFR312.82),  (2)  certain  end 
of  Phase  1  meetings  (21CFR312.82),  (3)  end  of  Phase  2/pre-Phase  3  meetings 
(21CFR312.47),  and  (4)  pre-NDA/BLA  meetings  (21CFR312.47). 
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ACRONYMS  AND  ABBREVIATIONS  FOR  APPENDIX  F 


510(k) 

Premarket  Notification  for  Medical  Devices 

ADR 

Adverse  Drug  Reaction 

BLA 

Biologies  Ficense  Application 

CBER 

Center  for  Biologies  Evaluation  and  Research 

CDER 

Center  for  Drug  Evaluation  and  Research 

CDRH 

Center  for  Devices  and  Radiologic  Health 

CFR 

Code  of  Federal  Regulations 

cGMP 

Current  Good  Manufacturing  Practices 

CMC 

chemistry,  manufacturing,  and  controls 

DMR 

Device  Master  Record 

DoD 

Department  of  Defense 

DSMICA 

Division  of  Small  Manufacturers,  International  and  Consumer 
Assistance 

FD&C 

Federal  Food,  Drug,  and  Cosmetic 

FDA 

Food  and  Drug  Administration 

GAO 

U.S.  General  Accounting  Office 

GCP 

Good  Clinical  Practice 

GFP 

Good  Faboratory  Practice 

HFD 

An  FDA  mailing  address  (e.g.,  HFD-970) 

HIV 

Human  Immunodeficiency  Virus 

HW/SW 

hardw  are/softw  are 

ICH 

International  Conference  on  Harmonisation 

IDE 

Investigational  Device  Exemption 

IM/IT 

Information  Management/Information  Technology 

IND 

Investigational  New  Drug  Application 

IOT&E 

initial  operational  test  and  evaluation 

MAIS 

Major  Automated  Information  System 

MATT 

Manufacturers  Assistance  and  Technical  Training 

MDA 

Milestone  Decision  Authority 

MDAP 

Major  Defense  Acquisition  Program 

NARA 

National  Archives  and  Records  Administration 

NASA 

National  Aeronautics  and  Space  Administration 

NDA 

New  Drug  Application 

OPA 

Office  of  Premarket  Approval 

PAI 

Preapproval  Inspection 

PI 

principal  investigator 

PM 

program  manager 

PMA 

Premarket  Approval 

POC 

point  of  contact 

QSIT 

Quality  System  Inspection  Technique 

QSR 

Quality  System  Regulation 

R&D 

research  and  development 
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RDT&E 

S&T 

SAE 

SE 

T&E 

TRL 

use 

USAMRMC 


research,  development,  test  and  evaluation 

science  and  technology 

Serious  Adverse  Event 

Substantial  Equivalence 

test  and  evaluation 

Technology  Readiness  Level 

United  States  Code 

United  States  Army  Medical  Research  and  Materiel  Command 
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APPENDIX  G 

MANUFACTURING  READINESS 


Matters  of  manufacturing  readiness  and  producibility  are  as  important  to  the  suc¬ 
cessful  development  of  a  system  as  those  of  readiness  and  capabilities  of  the  technologies 
intended  for  the  system.  Their  importance  has  long  been  recognized  in  Department  of 
Defense  (DoD)  acquisition  and  it  is  currently  reflected  in  Department  of  Defense  Directive 
(DoDD)  5000.1.  Specifically,  from  DoDD  5000.1: 

E.1.14  Knowledge-Based  Acquisition.  PMs  shall  provide  knowledge  about 
key  aspects  of  a  system  at  key  points  in  the  acquisition  process.  PMs  shall 
reduce  technology  risk,  demonstrate  technologies  in  a  relevant  environment, 
and  identify  technology  alternatives,  prior  to  program  initiation.  They  shall 
reduce  integration  risk  and  demonstrate  product  design  prior  to  the  design 
readiness  review.  They  shall  reduce  manufacturing  risk  and  demonstrate 
producibility  (emphasis  added)  prior  to  full-rate  production. 

In  accordance  with  this  policy,  Department  of  Defense  Instruction  (DoDI  5000.2) 
specifies  the  requirements  for  assessing  or  demonstrating  the  manufacturing  readiness  of  a 
system  at  various  stages  of  its  development. 

Currently,  standard  methods  and  metrics  similar  to  Technology  Readiness  Levels 
(TRLs)  do  not  exist  to  characterize  the  status  of  the  manufacturing  readiness  of  develop¬ 
mental  systems.  Best  industry  and  government  practices  have  shown  that  most  significant 
cycle  time  reductions  result  when  technology  performance  and  manufacturing  processes  are 
matured  concurrently.  A  Transition  Working  Group,  comprised  of  representatives  from  the 
Military  Services,  Defense  Logistics  Agency  (DLA),  Missile  Defense  Agency  (MDA),  and 
industry,  recently  addressed  the  issue  of  rapid,  affordable  transition  of  technology  to  acqui¬ 
sition  and  generated  a  set  of  definitions  and  descriptions  for  Manufacturing  Readiness  Lev¬ 
els  (MRLs).  MRLs  are  measures  used  to  assess  the  system  engineering/design  process  and 
maturity  of  a  technology’s  associated  manufacturing  processes.  This  appendix  provides  the 
definitions  and  descriptions,  with  a  view  toward  improving  awareness  of  the  MRL  concept 
and  for  use  of  the  MRLs  during  system  research  and  development  (R&D). 

Table  G-l  shows  the  proposed  MRLs.  Nine  levels  of  MRLs  were  used  for  conven¬ 
ience  when  comparing  technology  readiness  and  manufacturing  readiness.  Lor  example,  an 
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Table  G-1.  MRL  Definitions  and  Descriptions 


MRL 

Definition 

Description 

Acquisition  Phase 

1 

NA 

2 

NA 

3 

Manufacturing 
concepts  identified 

Identification  of  current  manufacturing  concepts  or 
producibility  needs  based  on  laboratory  studies. 

Pre-concept  refinement 

4 

Laboratory 

manufacturing 

processes 

identified 

Key  processes  identified  and  assessed  in  laboratory. 

Risk  mitigation  strategies  identified  to  address 
manufacturing/producibility  shortfalls.  Preliminary  Cost  as 
an  Independent  Variable  (CAIV)  targets  set  and  cost 
drivers  identified. 

Concept  refinement  leading  to 
a  Milestone  A  decision 

5 

Manufacturing 

process 

development 

Trade  studies  and  laboratory  experiments  result  in 
development  of  key  manufacturing  processes  and  initial 
sigma  levels  needed  to  satisfy  CAIV  targets.  Preliminary 
manufacturing  assembly  sequences  identified.  Process, 
tooling,  inspection,  and  test  equipment  in  development. 
Significant  engineering  and  design  changes.  Quality  and 
reliability  levels  not  yet  established.  Tooling  and  machines 
demonstrated  in  the  laboratory.  Physical  and  functional 
interfaces  have  not  been  completely  defined. 

Technology  Development  (TD) 

6 

Critical 

manufacturing 

processes 

demonstrated 

Critical  manufacturing  processes  initially  demonstrated 
for  the  relevant  environment  (laboratory  or  simulated 
operational  environment).  Initial  goals  established  for 
yields.  Process  and  tooling  generally  mature.  Frequent 
design  changes  still  occur.  Investments  in  machining  and 
tooling  identified.  Quality  and  reliability  levels  identified. 
Design  to  cost  goals  identified. 

TD,  leading  to  a  Milestone  B 
decision 

7 

Prototype 

manufacturing 

system 

Prototype  system  built  based  on  mature  tooling.  Initial 
sigma  levels  established,  based  on  yields  and  quality  data 
from  laboratory  or  simulations.  Design  changes  decrease 
significantly.  Process  tooling  and  inspection  and  test 
equipment  demonstrated  in  pre-production  environment. 
Manufacturing  processes  well  understood.  CAIV  and 
design  to  cost  goals  validated. 

System  Development  and 
Demonstration  (SDD) 

8 

Manufacturing 
process  maturity 
demonstration 

Manufacturing  processes  demonstrate  acceptable  yield 
and  producibility  levels  for  pilot  line,  low  rate  initial 
production  (LRIP),  or  similar  item  production.  All  design 
requirements  satisfied.  Manufacturing  processes  well 
understood  and  controlled  to  4-sigma  or  appropriate 
quality  level.  Minimal  investment  in  machine  and  tooling 
(should  have  completed  demonstration  in  at  least  a  low- 
rate  production  environment).  Cost  estimates  less  than 

125  percent  of  cost  goals  (e.g.,  design-to-cost  and  CAIV 
goals  met  for  LRIP). 

SDD,  leading  to  a  Milestone  C 
decision  and  LRIP 

9 

Manufacturing 
processes  proven 

Manufacturing  line  operating  at  desired  sigma  or  similar 
quality  level.  Stable  design  and  production.  All 
manufacturing  processes  controlled  to  6-sigma  or 
appropriate  quality  level.  Cost  estimates  less  than 

110  percent  of  cost  goals  or  meet  cost  goals  (e.g.,  CAIV 
and  design-to-cost  goals  met). 

Production,  deployment,  and 
support 
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MRL  of  6  or  7  is  appropriate  for  entry  to  milestone  B  and  assessment  during  System 
Development  and  Demonstration  (SDD),  just  as  a  TRL  of  6  or  7  is  appropriate  for  Mile¬ 
stone  B  and  SDD.  MRLs  1  and  2,  because  the  relative  immaturity  of  technology  at  TRL  1 
and  2,  are  not  seen  as  meaningful  for  assessing  manufacturing  readiness. 

The  MRL  definitions  are  based  on  the  integration  of  existing  industry,  government 
agency,  and  technical  coalition  standards  and  recommendations  to  address  producibility 
concerns  earlier  in  the  development  phase  (e.g.,  Engineering  &  Manufacturing  Readiness 
Levels  being  used  by  the  Milestone  Decision  Authority  (MDA)].  They  can  be  tailored  to 
apply  at  the  component,  subsystem,  and  system  levels  and  are  a  framework  with  specific 
criteria  and  metrics  to  capture  the  design  and  manufacturing  knowledge  for  product  devel¬ 
opment,  demonstration,  and  production.  The  definitions  support  assessments  of  the  maturity 
of  the  design,  related  materials,  tooling,  test  equipment,  manufacturing  processes,  quality  and 
reliability  levels,  and  key  characteristics  necessary  for  producible  and  reliable  products. 
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ACRONYMS  AND  ABBREVIATIONS  FOR  APPENDIX  G 


CATV 

Cost  as  an  Independent  Variable 

DLA 

Defense  Logistics  Agency 

DoD 

Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

LRIP 

low  rate  initial  production 

MDA 

Missile  Defense  Agency 

MRL 

Manufacturing  Readiness  Level 

R&D 

research  and  development 

SDD 

System  Development  and  Demonstration 

TD 

Technology  Development 

TRL 

Technology  Readiness  Level 
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ELEMENTS  OF  TECHNOLOGY  TRANSITION  AGREEMENT 


ELEMENTS  OFTETHMOLOGV  TRANSITION  AGREEMENT 

I  lif  fullu*i«i£fi™if»L5  JhwU  tw-fG*SKlemi  for  lmu$|i»  la  a  Iwhanlo^y  ifcmiiwiK 
bctwcM  nn>  At<iuk5lil*n  iungiAiiL,  chf  ncdvtr  a  1«hnolo|a'  Or  n 'liability 

t1tTfki|nii-rri[.  ind  a  science  burl  lerbiMilagy  activity.,  (he  rifveJogier  uml  pi  multi-  orrlit 
itrhiMlngy.  Soi  tvcry  ■:  no  af  ih  rst  elements  is  j jipr ■;  |i  riile  for  titr  agreement,  h  ue 
each  iprermrnt  should  h*vr  cun  side  red  these  far  inclusion, 

Agrremenli,  fa  be  effective,  mu  si  In  reviewed  periDdftilLywith  bw(h  5&T  1 1 ~  ri 1 1  a. i ■  l- 1 1 1  ■■  _~i l 
a/iri  |i  i  m j.’ r nm  office  run  in  rr  men  I  represent  tl  iver  part  ir  ip  ?  I  in;;.  Three-  reviews  s  li.jiilo 
add  rcss  lfrlm»t*3  And  future  direttitwi. 


ftlrmenli  Id  br  pru-vidcal  b>  thr  Program  -TJITirr: 

a.  Tpi-fid-r  AHjuktlcIwi  traginin.  A  brief  dtsaiplirm  rf  she  icqn isitirsn  pi  ogl  him 

i  mended  [a  receive  :hc  Icchnc'iocy  thz.1  is  1o  rrutiilMned.  include  nujct  pm^rieo 
objectives,  Anstrtl  pMst  rf KifcuiicifM  lift  cyclt,  And  pojKttd  irwlMf  -aprnLianel 
laipAbiiiflJ  rUco 

be  ]' rflf  ra m  M  an »i;rr'Pruj rtC  Q ITicer.  Piojp-um  manapi:i  imd  individual  m  pre-g^am 
nJliCC  ift^poi:s.ihie  ftrday-co-rlth  hinnaocmerr:  with  mmaci  informal  ion. 

c.  Aci|uiiilr»n  Fragrant  V«bnol*ji[y  Nrtrt.  Hrftfdcscriprioti  in$  beavtdlt  iftiL  this 
ctchnolfiQ.'  will  bring  lolhe  loquisicinci  jtrngnun,  < r  need  suisfeed  Where  possible, 
rclclu  be no In  tfr  QRU,  KTP,  do  [mfkidc  rwd  dales  fta  specific  -capabilities. 

d.  infegralinn  Strategy  Describe  Che  process  fee  intubating  Ihe  cectuKilogy  info  lb: 
acquisition  piC^nm  Include  tftmtnti  id' oi-uu:  siliLT  Jlralijuy  -  tviHii  tfoodry 
Acquisition.  bftvdk  upgrade,  „  15  wdl  u  requited  cootHKHtf  to  eofttSCitir 
AflTCiaifttivIs 

Ekmenfi  If  hf  nrgyrdvd  fry  S&T  Activity 

h.  Description  of  T'ccbu-ulug^ -or  Capability  Lu  Ibr  Delivered.  Brief  descripLian  of 
■Mi a i  ihi"  ijAT  activity  inlands  Co  develop  fttf  IrimiBai »  iIk  aCQuis-'iiuii  prupam. 
Include  CApkbilhy  delivery  dales 

ft  Technology  Manager.  individual  desigpuLed  by  the  5&T  nclivhy  lobe Ihe 

Viax’d'-ni.'ui  mid  'Jay-ro-diy  manage'  -of  Lhc  dP-efupmml  of  the  needed  Lechnniipy. 

c.  Cnrrenl  SlAlhJ  orfTetiihOlPftv, 

1.  Stilus  Summitry  StfflHrtribt  current  slaLe  ordrvelopmcrV!.  Identify  piimry 
4rccS  whcr«  adi'ii  i4ri«|  drvdOpitVHin  is  ioquirod.  Pr&viSe  mnnalc  of oumul  THL. 
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>.  RJih  Analysis.  Major  ircis  nf  risk,  pnarilLzed,  J-iUi  planned  mih£fflion  AEriviiics 
Juliiln-  technical  (c.g...  prenluoitolity,  EjffL>r  Jal'iMy,  SUftWAhilrty)  onst,  rnd 
schedule  risks. 

d.  lech  iioIvk*'  HMHflpiiiMic  Suilep)  U.il!  ne  ipwoidi  p.mm-J.  Jclli>>ts  i  vc|ui>  cH 
bcyvrai  l^wfft  currculy  underway;  inle^ral -ran  plans  Tmu  tiplf  prons-ES  ai«  plm'iNdd 
Planned  ATI)  w  AL  l  U  devulupmenls,  inapplicable 

t  |Cty  Technical  IVlMiura <il  kudinus  laTrinsilian.  Identify  the  key  parimtlCTS 
Pi  flirntNiEts  that  will  be  used  bn  measure  whether  Of  1HH  (foe  1«JincJofy  devefripnaeat 
tff(m  i:  pruraafing  appPOfUWEdy.  Include  purimtier  Its  be  tricked,  cunenl  stale, 

'  interim  prapeji  ettitHMtSL  Jfid  fmsl  cftjeaivie  Technology  Readiness  Levels  art  a 
irjeaLLie  of  technical  rrarurily  and  can.  be  used  Id  assess  leadinGil  to  U  ltsiI'uii. 

t  iFwigram  Finn,  Show  majw  acliYiiicsKflbn*  Damprised  by  the  lechnolngy 
development  activity  *iih  mileacncs 

ISignatures.  TkIwuIckj  irfthtitifti:  a^ceniciiin  should  be  signed  as  required  Id  commit 
Ibe  parlitiiMting  (irpniialions  bn  Ike  ptm  nut'  mv  J  in  ihe  qpcemsnl  The  program 
martueaf^tif  Ihe  acquisiLkm  prOgftnifS)  lilVGivCd  *nd  iheEST  prefect  manager.  should 
sigp 
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-  SAMPLE  - 


TECKlNOiJWV  TRANSIT  K>N  ACkEEMENT 

Itnif  Traji<jliiiL  AgJfcMlKIll 

1.  Dfscriptiun  of  TrchncilOg}  crCipalHliljr  L*  t*c  UtlfvtrtA. 

2.  'LiEf^^l  Arqoiiitiotn  Program, 

3.  Ati'jii  isilstn  Frajjn  m  Tfdinof+gy  Nwil 

4.  I  nrc-pn  liun  SriAltfiJ 

5.  I'l  Mgi  yiji  .'jjruqEr-’PrGjfrl  Oflinr  S.  Tfchnofoiyf  M»n«qrr 

Tirlnical  IfrtjM-ilN  Jiul  lKiiiUTnnimj|irE 

1,  '  I  Vir  ti  1 1  ■=■!  Ij  g:h  _  Curnenrt.  Slalb? 


j.  5u  nr,  miry  -  Slaliu 
lb,  PfcMnityiu 


To  Kiak* 

Brief  DtimuiLui 
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1.  Tetimnlnp1  Ik^flnpnwfit  Sci-Hegy, 


j,  Key  tir  iTDBsilipn  ltntiJii*tti 


AHnUulr.T'jcamMtl 

t'umnl 

]UI<-h»  lenity  LHh=) 

Foil  OSXrcM 

- -J 

- ■-  ■  — 

lLnijLiam  flin 


SIGNATURES: 


Ac  i|iii:  ilivn  I'M 


SAT  ]'Fpj  tee  M  in iccr 


ACRONYMS  AND  ABBREVIATIONS  FOR  APPENDIX  H 


ACTD 

Advanced  Concept  Technology  Demonstration 

ATD 

Advanced  Technology  Demonstration 

FY 

Fiscal  Year 

KPP 

key  performance  parameter 

ORD 

Operational  Requirements  Document 

S&T 

science  and  technology 

TRL 

Technology  Readiness  Level 
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